Case study

Building a Tweet Sentiment Tracker on Public Social Data

A personal project combined collection, storage, and sentiment scoring to track how tone changed across a large public tweet history.

Published Mar 23, 2026 Updated Mar 24, 2026

Stack

Python, NLTK, PostgreSQL, and AWS infrastructure.

Overview

The project focused on building the pipeline, not just the score.

Context

I wanted to analyze how tone shifted across a large public tweet history. The interesting part was not only the sentiment output, but also the full workflow required to collect, store, and process the text consistently.

Public social data tends to look simple from far away, but it becomes a data engineering problem as soon as the collection and storage steps matter.

Approach

I collected historical tweets, stored them in PostgreSQL on AWS, and used VADER sentiment scoring through NLTK to build a time-based view of emotional change.

The project was useful because it linked text analysis to a concrete storage and retrieval layer instead of treating the scoring step in isolation.

Workflow

Collection, storage, and scoring needed to work as one system.

Collection

Historical tweets were gathered and normalized so they could be processed in a consistent format.

Storage

PostgreSQL on AWS provided a practical way to store and query the tweet history once the dataset moved beyond a simple local experiment.

Sentiment scoring

VADER sentiment scoring was applied to create a simple time-based signal of how the tone shifted over the corpus.

Results

The main output was a usable text-analysis pipeline.

Data

Public tweets

Used a large public tweet history as the source dataset.

Storage

PostgreSQL

Stored the history in a database instead of keeping the project notebook-only.

Analysis

VADER

Used a lightweight sentiment approach to create a trackable trend signal.

Takeaways

Text analysis becomes more credible when the data workflow is explicit.

What mattered

The durable part of the project was the end-to-end workflow from collection to storage to scoring, not the sentiment number by itself.

Why it was useful

It made text analysis feel like a repeatable data pipeline rather than a one-off demo on a static file.

Continue with connected pages

Case studies and methods that connect to the same operational questions.

Case study

Supplement Recommendation Prototype

Built a web prototype that matched supplement suggestions to profile inputs using scraped product and review data.

End-to-end prototype from data collection to recommendation flow

Case study

Sales Forecasting Competition

Won a public forecasting competition by matching the preprocessing and forecast logic to the competition's asymmetric scoring rule.

First place by matching preprocessing and forecasts to the scoring rule

Back to all work