{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "markdown",
"source": [
"# Project Description"
],
"metadata": {
"id": "hOeAjWK7tc2S"
}
},
{
"cell_type": "markdown",
"source": [
"As described in the accompanying document, you were recently hired as part of an independent, non-partisan commission that is analyzing the emotional stability of billionaries. The goal of the commission is to asssess their suitability for positions in government office. Many sources of data are being considered, and you are the head of a sub-committee that was tasked with investigating the tweets of Elon Musk.\n",
"\n",
"**You are strongly encouraged to use generative AI for this project, with no constraints.**"
],
"metadata": {
"id": "iDo5A2VItfwM"
}
},
{
"cell_type": "markdown",
"source": [
"# Starter Code\n",
"\n",
"The following code may prove useful for your goals."
],
"metadata": {
"id": "a1CraMUTZmUg"
}
},
{
"cell_type": "markdown",
"source": [
"## Download the tweets:"
],
"metadata": {
"id": "bCCNO3UEzjU7"
}
},
{
"cell_type": "code",
"source": [
"import pandas as pd\n",
"\n",
"# url address for Musk's tweets\n",
"url = 'https://raw.githubusercontent.com/MainakRepositor/Datasets/refs/heads/master/Elon%20Tweets/2018.csv'\n",
"\n",
"# load into a df\n",
"df = pd.read_csv(url, index_col=0)\n",
"\n",
"# tweet contents as a dictionary\n",
"elon_tweets = df['tweet'].to_dict()\n",
"\n",
"# features to keep\n",
"feature_keep_list = [\n",
" 'id',\n",
" 'conversation_id',\n",
" 'created_at',\n",
" 'date',\n",
" 'tweet',\n",
" 'day',\n",
" 'hour',\n",
" 'retweet',\n",
" 'nlikes',\n",
" 'nreplies',\n",
" 'nretweets'\n",
" ]\n",
"\n",
"cols_to_keep = [df.columns.get_loc(col) for col in feature_keep_list]\n",
"df = df.iloc[:, cols_to_keep]"
],
"metadata": {
"id": "85DJQBVGzlyq"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"df.shape"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "AJ6gUCv7Z23i",
"outputId": "c4122802-e133-49b3-e4f6-f54de691047d"
},
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"(2285, 11)"
]
},
"metadata": {},
"execution_count": 4
}
]
},
{
"cell_type": "markdown",
"source": [
"## A bit of cleanup to make for easier analysis:"
],
"metadata": {
"id": "Rcg_uU4ebjOx"
}
},
{
"cell_type": "code",
"source": [
"# Function that cleans text\n",
"def clean_text(unformatted_string):\n",
" cleaned_words = []\n",
" words = unformatted_string.split()\n",
" for word in words:\n",
" if not word.startswith('@') and not word.startswith('#') and not word.startswith('http'):\n",
" cleaned_words.append(word)\n",
" formatted_string = ' '.join(cleaned_words).lower().rstrip('?!.')\n",
" return formatted_string\n",
"\n",
"# How to create a column of cleaned tweets\n",
"df['cleaned_tweets'] = df['tweet'].apply(clean_text)\n",
"\n",
"# Extracting the month from a date-string\n",
"def extract_month(date):\n",
" month = date.split()[0].split('-')[1]\n",
" return month\n",
"\n",
"# How to create a column with the month\n",
"df['month'] = df['date'].apply(extract_month)"
],
"metadata": {
"id": "TEQQFIJoaWqH"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"df.head()"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 434
},
"id": "hFh1rH6ZbLr-",
"outputId": "f824937b-451b-4845-a3a4-8940151bf40f"
},
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
" id conversation_id created_at \\\n",
"0 1079469237230493696 1079155749077475328 1.546200e+12 \n",
"1 1079459047252709377 1079457030060924928 1.546198e+12 \n",
"2 1078716366591483904 1078716366591483904 1.546021e+12 \n",
"3 1078529476018008064 1078529330492321793 1.545976e+12 \n",
"4 1078395092090699776 1078383289356902401 1.545944e+12 \n",
"\n",
" date tweet \\\n",
"0 2018-12-30 20:08:16 @Ben757677 @vincent13031925 Thanks Tesla owner... \n",
"1 2018-12-30 19:27:46 @Grimezsz Turns out if you take Dayquil, you e... \n",
"2 2018-12-28 18:16:38 Excited to have Larry Ellison & Kathleen W... \n",
"3 2018-12-28 05:53:59 @NutmegTheRed @Tesla Ok \n",
"4 2018-12-27 21:00:00 @westcoastbill Tesla team did a great job! \n",
"\n",
" day hour retweet nlikes nreplies nretweets \\\n",
"0 7 20 False 2789 99 132 \n",
"1 7 19 False 14452 303 940 \n",
"2 5 18 False 36045 1013 2158 \n",
"3 5 5 False 535 40 17 \n",
"4 4 21 False 3750 100 141 \n",
"\n",
" cleaned_tweets month \n",
"0 thanks tesla owners in china! looking forward ... 12 \n",
"1 turns out if you take dayquil, you exit the ma... 12 \n",
"2 excited to have larry ellison & kathleen w... 12 \n",
"3 ok 12 \n",
"4 tesla team did a great job 12 "
],
"text/html": [
"\n",
"
| \n", " | id | \n", "conversation_id | \n", "created_at | \n", "date | \n", "tweet | \n", "day | \n", "hour | \n", "retweet | \n", "nlikes | \n", "nreplies | \n", "nretweets | \n", "cleaned_tweets | \n", "month | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "1079469237230493696 | \n", "1079155749077475328 | \n", "1.546200e+12 | \n", "2018-12-30 20:08:16 | \n", "@Ben757677 @vincent13031925 Thanks Tesla owner... | \n", "7 | \n", "20 | \n", "False | \n", "2789 | \n", "99 | \n", "132 | \n", "thanks tesla owners in china! looking forward ... | \n", "12 | \n", "
| 1 | \n", "1079459047252709377 | \n", "1079457030060924928 | \n", "1.546198e+12 | \n", "2018-12-30 19:27:46 | \n", "@Grimezsz Turns out if you take Dayquil, you e... | \n", "7 | \n", "19 | \n", "False | \n", "14452 | \n", "303 | \n", "940 | \n", "turns out if you take dayquil, you exit the ma... | \n", "12 | \n", "
| 2 | \n", "1078716366591483904 | \n", "1078716366591483904 | \n", "1.546021e+12 | \n", "2018-12-28 18:16:38 | \n", "Excited to have Larry Ellison & Kathleen W... | \n", "5 | \n", "18 | \n", "False | \n", "36045 | \n", "1013 | \n", "2158 | \n", "excited to have larry ellison & kathleen w... | \n", "12 | \n", "
| 3 | \n", "1078529476018008064 | \n", "1078529330492321793 | \n", "1.545976e+12 | \n", "2018-12-28 05:53:59 | \n", "@NutmegTheRed @Tesla Ok | \n", "5 | \n", "5 | \n", "False | \n", "535 | \n", "40 | \n", "17 | \n", "ok | \n", "12 | \n", "
| 4 | \n", "1078395092090699776 | \n", "1078383289356902401 | \n", "1.545944e+12 | \n", "2018-12-27 21:00:00 | \n", "@westcoastbill Tesla team did a great job! | \n", "4 | \n", "21 | \n", "False | \n", "3750 | \n", "100 | \n", "141 | \n", "tesla team did a great job | \n", "12 | \n", "