Introduction

Imagine you work for a travel company and you want to predict whether a particular trip booking will fall into one category or another — maybe “premium” versus “standard,” or “likely to cancel” versus “likely to keep.” With 160,000 historical bookings and 50 different pieces of information about each one, manually sorting through the data would be impossible. This is exactly the kind of problem that machine learning can solve.

In this case study, we walk through a complete data science workflow: exploring the data, cleaning it up, and then building predictive models that can automatically classify future bookings. The entire project was carried out in Python, and we will break down each step in plain language so anyone can follow along.

What Does the Data Look Like?

Our dataset contains 160,000 records with 50 input features and 1 outcome label (called “y”). The outcome is binary — it takes only two values, 0 or 1 — making this a classification problem, where the goal is to assign each record to one of two groups.

Out of the 50 input features:

• 45 are numeric — continuous numbers like measurements or scores.
• 5 are categorical — text-based labels describing things like geography, time, and percentages.

The five categorical features turned out to be quite informative:

Feature	What It Represents	Example Values
x24	Continent	Asia, Europe, America
x29	Month	June, July, August
x30	Day of the week	Monday through Friday
x32	Percentage change	0.0%, 0.01%, -0.01%
x37	Dollar amount	$1313.96, $-620.66

Looking at these features together, the data appears to describe cross-continent summer holiday travel activity — most events occur in Asia during summer months and on weekdays.

Data Preview

Below is a scrollable preview of the first 10 rows (out of 160,000). Scroll horizontally to see all 51 columns.

⬇ Download the full dataset (final_project.csv.tar.gz)

Showing 5 of 160,000 rows × 51 columns. Scroll right to see all features.

Step 1: Handling Missing Data

Real-world data is rarely perfect. When we checked for missing information, we found that many features had small gaps — but none exceeded 0.03% of all records. Importantly, the outcome label had no missing values at all.

Figure 1: The percentage of missing values across all features. The gaps are tiny — well under 0.05% for every feature.

Because the amount of missing data was so small, we used two simple strategies to fill in the blanks:

• For numeric features, we used a technique called K-Nearest Neighbors (KNN) Imputation. In simple terms, when a value is missing, the algorithm looks at the five most similar records and fills in the gap using their average.
• For categorical features, we simply used the most frequently occurring value as the replacement.

Step 2: Understanding the Outcome Distribution

Before building any model, we wanted to know how balanced our two outcome groups were. It turned out that about 60% of records belonged to class 1 and 40% to class 0.

Figure 2: The split between the two outcome groups (0 and 1). A 40/60 split is well-balanced, meaning neither group overwhelms the other.

This is good news — when one group is far larger than the other (say 99% vs. 1%), models can “cheat” by always guessing the bigger group. A 40/60 split avoids that trap.

Step 3: Exploring the Categorical Features

We dug deeper into the four text-based features to see what stories they told:

Figure 3: Bar charts showing how the categorical features break down by outcome group.

Key takeaways:

• Continent (x24): A striking 85% of records originate from Asia, with only 10% from Europe and 5% from America.
• Month (x29): Summer months (June, July, August) dominate — together they account for over 75% of all records.
• Day of week (x30): Activity is concentrated on weekdays, peaking on Wednesday. Weekends show virtually zero activity, and Monday/Friday have very low counts.
• Percentage change (x32): About 90% of values fall in the narrow range of 0.0% to 0.01%, indicating very small fluctuations.

These patterns are consistent with seasonal business travel — heavy during summer, concentrated on core weekdays, and centered in Asia.

Step 4: Checking Relationships Among Features

When features are highly correlated (i.e., they essentially carry the same information), including both can confuse or slow down models without adding value. We computed the correlations among all 46 numeric features and found that only 4 pairs had a correlation above 0.7.

Figure 4: A heatmap of correlations among the numeric features flagged by VIF analysis. Darker red or blue squares indicate stronger relationships.

Two pairs stood out — (x2, x6) and (x38, x41) — so we removed one feature from each pair (x6 and x41) to reduce redundancy before training some of our models.

Step 5: Visualizing the Numeric Features

We also plotted histograms for all numeric features to check their shape. Every single one displayed a bell-shaped (Gaussian) distribution, which is a sign of well-behaved data.

Figure 5: Histograms for all numeric features, color-coded by outcome group. The bell-curve shape is consistent across the board.

Because we planned to use tree-based models (which split data based on thresholds rather than distances), we did not need to rescale these features to a common range — trees naturally handle different scales.

Step 6: Building the Models

We trained four models using two popular machine learning algorithms:

1. Random Forest (RF) — an ensemble method that builds many decision trees and combines their votes.
2. XGBoost — a more advanced boosting algorithm that builds trees sequentially, with each new tree correcting the mistakes of the previous ones.

For each algorithm, we trained two versions:

Model	Features Used
RF Full	All features
RF Selected	Reduced features (correlated ones removed)
XGBoost Full	All features
XGBoost Selected	Reduced features

How Did They Perform?

We evaluated each model on a held-out test set — data the models had never seen during training — using two key metrics:

• Accuracy: What percentage of predictions were correct overall?
• AUC (Area Under the ROC Curve): How well can the model distinguish between the two groups? An AUC of 1.0 means perfect separation; 0.5 means no better than flipping a coin.

Step 7: Comparing the Results

The ROC curve below tells the full story:

Figure 6: ROC curves for all four models. The closer a curve hugs the top-left corner, the better the model is at distinguishing between the two groups.

Model	Accuracy	AUC
RF Full	96%	High
RF Selected	95%	Lower
XGBoost Full	97%	Highest
XGBoost Selected	95%	Lower

The XGBoost model trained on all features was the clear winner, achieving 97% accuracy and the highest AUC score. This tells us two things:

1. XGBoost outperformed Random Forest in this scenario, likely because its sequential correction mechanism captures subtle patterns that independent trees miss.
2. Using all features was better than the reduced set, meaning even the correlated features contributed useful information that improved predictions.

What Makes a Feature Important?

One of the advantages of tree-based models is that they can tell us which features matter most. The chart below shows the feature importances from the best-performing XGBoost model:

Figure 7: Feature importance scores from XGBoost. Taller bars indicate features that had more influence on the model’s decisions.

This kind of insight is valuable beyond just prediction — it helps the business understand what factors drive the outcome, enabling smarter decisions about data collection, strategy, and resource allocation.

Key Takeaways

1. Data quality matters. Even though the dataset had missing values, they were so few (under 0.03%) that simple imputation techniques worked well. Always check your data before jumping to modeling.

2. Exploratory analysis reveals hidden stories. By simply plotting the categorical features, we uncovered that the data likely represents summer business travel in Asia — context that can guide both model design and business interpretation.

3. Feature engineering has limits. Removing correlated features actually hurt model performance. Tree-based models are already good at handling redundancy, so aggressive feature pruning may do more harm than good.

4. XGBoost is a powerful default choice. For tabular data (rows and columns, like spreadsheets), XGBoost consistently ranks among the top-performing algorithms. In this study it delivered 97% accuracy with minimal tuning.

5. Model interpretability is a bonus. Feature importance plots let us peek inside the “black box” and understand which inputs the model relies on — a critical step for building trust and driving action.

Conclusion

This case study demonstrated a complete machine learning pipeline — from raw data to validated predictions — using Python’s scikit-learn and XGBoost libraries. The best model (XGBoost with all features) achieved an impressive 97% accuracy, showing that with careful data preparation and the right algorithm, we can build reliable predictive systems even from anonymized datasets.

Whether you are classifying travel bookings, predicting customer behavior, or sorting through medical records, the workflow is remarkably similar: explore, clean, model, evaluate, and interpret. The tools change, but the process stays the same.