{ "cells": [ { "cell_type": "code", "execution_count": null, "id": "dffa60e8", "metadata": {}, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "id": "3340eabf", "metadata": { "cell_marker": "\"\"\"" }, "source": [ "Inclass-Lab\n", "\n", "Automatically generated by Colab.\n", "\n", "Original file is located at\n", " https://colab.research.google.com/drive/13N7lQHvv4_LxKgcerumJ-18e5GfdBggm" ] }, { "cell_type": "code", "execution_count": null, "id": "34271feb", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "9491847c", "metadata": { "cell_marker": "\"\"\"" }, "source": [ "In-Class Lab: Comparing Classification Models (Breast Cancer Dataset)\n", "\n", "Learning Objectives\n", "\n", "---\n", "\n", "\n", "By the end of this lab, you should be able to:\n", "\n", "\n", "\n", "* Train multiple models (KNN, Logistic Regression, Decision Tree)\n", "* Compare performance across various scenarios\n", "* Understand the effect of scaling and evaluation methods\n", "* Interpret confusion matrix & classification report\n", "\n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "id": "cceb43dc", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 1:" ] }, { "cell_type": "markdown", "id": "818bfdad", "metadata": { "cell_marker": "'''", "lines_to_next_cell": 0 }, "source": [ "Load dataset\n", "Explain features/target\n", "Do train_test_split" ] }, { "cell_type": "markdown", "id": "6de94b5d", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [] }, { "cell_type": "code", "execution_count": null, "id": "a8899ca0", "metadata": {}, "outputs": [], "source": [ "from sklearn.datasets import load_breast_cancer\n", "data = load_breast_cancer()\n", "X = data.data\n", "y = data.target\n", "\n", "print(data.target_names)\n", "print(X.shape)\n", "#0 - malignant\n", "#1 - benigh" ] }, { "cell_type": "markdown", "id": "ae1b0d61", "metadata": { "cell_marker": "###############################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 2: IMPORT LIBRARIES FOR CLASSIFIERS\n", "##############################" ] }, { "cell_type": "code", "execution_count": null, "id": "8db6dfd6", "metadata": { "lines_to_next_cell": 0 }, "outputs": [], "source": [ "from sklearn.neighbors import KNeighborsClassifier\n", "from sklearn.linear_model import LogisticRegression\n", "from sklearn.tree import DecisionTreeClassifier\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "a2301b40", "metadata": { "cell_marker": "#########################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 3: IMPORT MODUL FOR DATA SPLIT\n", "########################################" ] }, { "cell_type": "code", "execution_count": null, "id": "75fe2532", "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split" ] }, { "cell_type": "markdown", "id": "fb69921d", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 4: SPLIT THE DATASET\n", "###################################" ] }, { "cell_type": "code", "execution_count": null, "id": "6eb66fc0", "metadata": {}, "outputs": [], "source": [ "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)" ] }, { "cell_type": "markdown", "id": "b182e930", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 5\n", "TASK 1: Train 3 models (no scaling)\n", "###################################" ] }, { "cell_type": "code", "execution_count": null, "id": "f31f0262", "metadata": {}, "outputs": [], "source": [ "knn = KNeighborsClassifier(n_neighbors=5)\n", "logreg = LogisticRegression(max_iter=5000)\n", "tree = DecisionTreeClassifier()\n", "\n", "knn.fit(X_train, y_train)\n", "logreg.fit(X_train, y_train)\n", "tree.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "id": "d3e0088e", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 6\n", "Task 2: Evaluate accuracy" ] }, { "cell_type": "markdown", "id": "3a8845a1", "metadata": { "cell_marker": "'''", "lines_to_next_cell": 0 }, "source": [ "* Which model performs best?\n", "* Are the results very different?" ] }, { "cell_type": "markdown", "id": "d49b62b3", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [] }, { "cell_type": "code", "execution_count": null, "id": "395d887c", "metadata": {}, "outputs": [], "source": [ "print(\"KNN:\", knn.score(X_test, y_test))\n", "print(\"LogReg:\", logreg.score(X_test, y_test))\n", "print(\"Tree:\", tree.score(X_test, y_test))" ] }, { "cell_type": "markdown", "id": "9de74c34", "metadata": { "cell_marker": "####################################" }, "source": [ "BLOCK 7\n", "Part 2: Evaluation Metrics\n", "Task 3: Confusion Matrix" ] }, { "cell_type": "code", "execution_count": null, "id": "deec64c6", "metadata": {}, "outputs": [], "source": [ "from sklearn.metrics import confusion_matrix\n", "print(confusion_matrix(y_test, knn.predict(X_test)))" ] }, { "cell_type": "markdown", "id": "18bc0c0e", "metadata": { "cell_marker": "#####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 8: CLASSIFICATION FREPORT" ] }, { "cell_type": "markdown", "id": "a2e8747f", "metadata": { "cell_marker": "'''", "lines_to_next_cell": 0 }, "source": [ "* Which model has better recall?\n", "* Which is better for detecting cancer?" ] }, { "cell_type": "markdown", "id": "88303af7", "metadata": { "cell_marker": "#####################################", "lines_to_next_cell": 0 }, "source": [] }, { "cell_type": "code", "execution_count": null, "id": "e9ef7bea", "metadata": {}, "outputs": [], "source": [ "from sklearn.metrics import classification_report\n", "print(classification_report(y_test, knn.predict(X_test)))" ] }, { "cell_type": "markdown", "id": "c1d8978a", "metadata": { "cell_marker": "########################################" }, "source": [ "BLOCK 9:\n", "Part 3: Scaling Effect\n", "Task 5: Apply scaling using pipeline\n", "#######################################" ] }, { "cell_type": "code", "execution_count": null, "id": "84e1f5d4", "metadata": {}, "outputs": [], "source": [ "from sklearn.pipeline import Pipeline\n", "from sklearn.preprocessing import StandardScaler\n", "from sklearn.metrics import accuracy_score, confusion_matrix, classification_report\n", "\n", "knn_scaled = Pipeline([\n", " ('scaler', StandardScaler()),\n", " ('knn', KNeighborsClassifier(n_neighbors=5))\n", "])\n", "\n", "logreg_scaled = Pipeline([\n", " ('scaler', StandardScaler()),\n", " ('logreg', LogisticRegression(max_iter=5000))\n", "])\n", "\n", "tree_scaled = Pipeline([\n", " ('scaler', StandardScaler()),\n", " ('tree', DecisionTreeClassifier(random_state=42))\n", "])\n", "\n", "knn_scaled.fit(X_train, y_train)\n", "logreg_scaled.fit(X_train, y_train)\n", "tree_scaled.fit(X_train, y_train)\n", "\n", "knn_pred_scaled = knn_scaled.predict(X_test)\n", "logreg_pred_scaled = logreg_scaled.predict(X_test)\n", "tree_pred_scaled = tree_scaled.predict(X_test)\n", "\n", "print(\"\\n=== RESULTS WITH SCALING ===\")\n", "print(\"KNN Accuracy:\", accuracy_score(y_test, knn_pred_scaled))\n", "print(\"Logistic Regression Accuracy:\", accuracy_score(y_test, logreg_pred_scaled))\n", "print(\"Decision Tree Accuracy:\", accuracy_score(y_test, tree_pred_scaled))" ] }, { "cell_type": "markdown", "id": "d5caa8ec", "metadata": { "cell_marker": "#####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 10: CLASSIFICATION REPORT\n", "####################################" ] }, { "cell_type": "code", "execution_count": null, "id": "6643f0af", "metadata": {}, "outputs": [], "source": [ "print(\"\\n=== CLASSIFICATION REPORTS (WITH SCALING) ===\")\n", "\n", "print(\"\\nKNN Report:\")\n", "print(classification_report(y_test, knn_pred_scaled))\n", "\n", "print(\"\\nLogistic Regression Report:\")\n", "print(classification_report(y_test, logreg_pred_scaled))\n", "\n", "print(\"\\nDecision Tree Report:\")\n", "print(classification_report(y_test, tree_pred_scaled))" ] }, { "cell_type": "markdown", "id": "c38c51d0", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 11: CONFUSION MATRIX\n", "###################################" ] }, { "cell_type": "code", "execution_count": null, "id": "444fbb13", "metadata": {}, "outputs": [], "source": [ "print(\"\\n=== CONFUSION MATRICES (WITH SCALING) ===\")\n", "\n", "print(\"\\nKNN Confusion Matrix:\")\n", "print(confusion_matrix(y_test, knn_pred_scaled))\n", "\n", "print(\"\\nLogistic Regression Confusion Matrix:\")\n", "print(confusion_matrix(y_test, logreg_pred_scaled))\n", "\n", "print(\"\\nDecision Tree Confusion Matrix:\")\n", "print(confusion_matrix(y_test, tree_pred_scaled))\n", "\n" ] }, { "cell_type": "markdown", "id": "c5d82a08", "metadata": { "cell_marker": "###############################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 12: MODWLS WITH CROSS-VALIDATION\n", "##############################################" ] }, { "cell_type": "code", "execution_count": null, "id": "17493cc3", "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import cross_val_score\n", "from sklearn.preprocessing import StandardScaler\n", "\n", "knn_pipeline = Pipeline([\n", " ('scaler', StandardScaler()),\n", " ('knn', KNeighborsClassifier(n_neighbors=5))\n", "])\n", "\n", "logreg_pipeline = Pipeline([\n", " ('scaler', StandardScaler()),\n", " ('logreg', LogisticRegression(max_iter=5000))\n", "])\n", "\n", "tree_model = DecisionTreeClassifier(random_state=42)\n", "\n", "knn_cv_scores = cross_val_score(knn_pipeline, X, y, cv=5, scoring='accuracy')\n", "logreg_cv_scores = cross_val_score(logreg_pipeline, X, y, cv=5, scoring='accuracy')\n", "tree_cv_scores = cross_val_score(tree_model, X, y, cv=5, scoring='accuracy')\n", "\n", "print(\"=== Cross-Validation Results ===\")\n", "\n", "print(\"\\nKNN\")\n", "print(\"CV Scores:\", knn_cv_scores)\n", "print(\"Mean Accuracy:\", np.mean(knn_cv_scores))\n", "print(\"Standard Deviation:\", np.std(knn_cv_scores))\n", "\n", "print(\"\\nLogistic Regression\")\n", "print(\"CV Scores:\", logreg_cv_scores)\n", "print(\"Mean Accuracy:\", np.mean(logreg_cv_scores))\n", "print(\"Standard Deviation:\", np.std(logreg_cv_scores))\n", "\n", "print(\"\\nDecision Tree\")\n", "print(\"CV Scores:\", tree_cv_scores)\n", "print(\"Mean Accuracy:\", np.mean(tree_cv_scores))\n", "print(\"Standard Deviation:\", np.std(tree_cv_scores))" ] }, { "cell_type": "markdown", "id": "94d94dd2", "metadata": { "cell_marker": "####################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 13: CONFUSION MATRIX\n", "###################################" ] }, { "cell_type": "code", "execution_count": null, "id": "c706abc3", "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import cross_val_predict\n", "from sklearn.metrics import confusion_matrix, classification_report\n", "\n", "knn_pred = cross_val_predict(knn_pipeline, X, y, cv=5)\n", "logreg_pred = cross_val_predict(logreg_pipeline, X, y, cv=5)\n", "tree_pred = cross_val_predict(tree_model, X, y, cv=5)\n", "\n", "print(\"\\n=== Confusion Matrices ===\")\n", "print(\"\\nKNN\")\n", "print(confusion_matrix(y, knn_pred))\n", "\n", "print(\"\\nLogistic Regression\")\n", "print(confusion_matrix(y, logreg_pred))\n", "\n", "print(\"\\nDecision Tree\")\n", "print(confusion_matrix(y, tree_pred))\n", "\n", "print(\"\\n=== Classification Reports ===\")\n", "print(\"\\nKNN\")\n", "print(classification_report(y, knn_pred))\n", "\n", "print(\"\\nLogistic Regression\")\n", "print(classification_report(y, logreg_pred))\n", "\n", "print(\"\\nDecision Tree\")\n", "print(classification_report(y, tree_pred))" ] }, { "cell_type": "markdown", "id": "34354ec9", "metadata": { "cell_marker": "###########################################", "lines_to_next_cell": 0 }, "source": [ "BLOCK 14: LETS GET THE BEST VALUE FOR K\n", "task: use the k value to recompute the previous KNN model and evaluate the performance\n", "#########################################" ] }, { "cell_type": "code", "execution_count": null, "id": "93fa45e5", "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import cross_val_score\n", "import numpy as np\n", "\n", "k_values = range(1, 21)\n", "scores = []\n", "\n", "for k in k_values:\n", " knn = KNeighborsClassifier(n_neighbors=k)\n", " cv_scores = cross_val_score(knn, X, y, cv=5)\n", " scores.append(np.mean(cv_scores))\n", "\n", "best_k = k_values[np.argmax(scores)]\n", "\n", "print(\"Best k:\", best_k)\n", "print(\"Best CV Accuracy:\", max(scores))\n" ] } ], "metadata": { "jupytext": { "cell_metadata_filter": "-all", "encoding": "# -*- coding: utf-8 -*-", "main_language": "python", "notebook_metadata_filter": "-all" } }, "nbformat": 4, "nbformat_minor": 5 }