Introduction

Superconductors are materials that can conduct electricity with zero resistance when cooled below a characteristic temperature known as the critical temperature ((T_c)). Discovering new superconductors and predicting their critical temperatures is a central challenge in materials science, as higher (T_c) materials could revolutionize energy transmission, magnetic levitation, and quantum computing.

In this study, we explore whether the critical temperature of a superconductor can be accurately predicted from its elemental and physical properties using regularized linear regression. Specifically, we compare Ridge Regression (L2 regularization) and Lasso Regression (L1 regularization) — two powerful techniques that address multicollinearity and perform automatic feature selection.

The Dataset

The dataset used in this study comes from a publicly available superconductor dataset and consists of two files:

1. train.csv — Physical Property Features

Property	Value
Samples	21,263 superconductor compositions
Predictors	81 features + 1 target
Target	critical_temp (in Kelvin)

The 81 predictors are derived from 8 fundamental physical properties of the constituent elements:

• Atomic Mass
• First Ionization Energy (fie)
• Atomic Radius
• Density
• Electron Affinity
• Fusion Heat
• Thermal Conductivity
• Valence

Each property is summarized across the elements in a given compound using 10 statistical aggregation functions: mean, weighted mean, geometric mean, weighted geometric mean, entropy, weighted entropy, range, weighted range, standard deviation, and weighted standard deviation. Together with number_of_elements, this yields 81 predictors.

2. unique_m.csv — Elemental Composition Features

This supplementary file contains 86 columns corresponding to individual chemical elements (H, He, Li, …, Rn), each representing the proportion or count of that element in the compound. It also includes the critical_temp target and a material column with the chemical formula (e.g., Ba0.2La1.8Cu1O4).

Target Variable Summary

Statistic	Value
Mean	34.42 K
Std. Dev.	34.25 K
Min	0.00021 K
25th Percentile	5.37 K
Median	20.00 K
75th Percentile	63.00 K
Max	185.00 K

Data Preview

Below is a scrollable preview of the first 3 rows of train.csv (out of 21,263). Scroll horizontally to see all 82 columns.

⬇ Download the full dataset (superconductor_data.tar.gz)

Showing 3 of 21,263 rows × 82 columns. Scroll right to see all features.

Exploratory Data Analysis (EDA)

Missing Values

A missing-value heatmap confirmed that neither dataset contains any missing values, which is a favorable starting point that eliminates the need for imputation strategies.

Figure 1: Missing-value heatmap for train.csv. The uniform color confirms zero missing values across all 21,263 rows and 82 columns.

Figure 2: Bar plots of missing value counts for train.csv (left) and unique_m.csv (right), both confirming no missing data.

Distribution of the Target Variable

The histogram of critical_temp revealed a right-skewed (positively skewed) distribution, with the majority of superconductors clustered at lower temperatures and a long tail stretching toward 185 K. This kind of skewed distribution can negatively affect the performance of linear models that assume normally distributed residuals.

Figure 3: Histogram of the target variable critical_temp, showing a pronounced right skew.

To address this, a Box-Cox power transformation was applied to the target variable. After transformation, the distribution became approximately symmetric and bell-shaped — much closer to the normal distribution assumption required by linear regression.

Figure 4: After Box-Cox transformation, the target distribution is approximately normal.

Feature Distributions

Histograms of all 81 predictors in train.csv were plotted on a 9×9 grid, color-coded by their statistical aggregation type:

Color	Aggregation Type
Cyan	mean
Green	weighted mean
Blue	geometric mean
Red	standard deviation
Orange	weighted std
Purple	entropy
Yellow	weighted entropy
Olive	range
Brown	weighted range

Figure 5: Histograms of all 81 predictors, color-coded by statistical aggregation type. Notice how features within each physical property group share similar distributional shapes.

This visualization revealed that features within the same physical property group (e.g., all 10 aggregations of atomic_mass) tend to have similar distributional shapes, hinting at high inter-feature correlation within groups.

Correlation Analysis

A Pearson correlation heatmap of all 81 predictors confirmed severe multicollinearity. Block-diagonal patterns were clearly visible, indicating that the 10 statistical aggregations within each physical property group are highly correlated with one another.

Figure 6: Pearson correlation heatmap for the 81 predictors in train.csv. The prominent block-diagonal structure reveals severe multicollinearity within each physical property group.

A separate correlation heatmap of the elemental composition features (from unique_m.csv) showed a more sparse correlation structure, as elemental presence tends to be more independent across compounds.

Figure 7: Pearson correlation heatmap for the elemental composition features in unique_m.csv, showing a much sparser correlation structure.

Variance Inflation Factor (VIF) Analysis

To quantify multicollinearity, the Variance Inflation Factor (VIF) was computed for every predictor. VIF measures how much the variance of a regression coefficient is inflated due to correlation with other predictors.

VIF Range	Interpretation
1 – 2	Very low collinearity; usually no concern
2 – 5	Moderate; often acceptable
5 – 10	High; coefficients may be unstable
> 10	Very serious multicollinearity

Initial VIF Results (All 81 Features)

The initial VIF analysis revealed extremely high VIF values across the board. For example:

Feature	VIF
wtd_mean_atomic_mass	818.37
wtd_gmean_atomic_mass	879.86
mean_atomic_mass	414.28
gmean_atomic_mass	444.20
wtd_entropy_Valence	307.31

These values are well above the threshold of 10, confirming that using all 81 features simultaneously in an ordinary linear regression would produce highly unstable coefficient estimates.

Figure 8: VIF bar plots for train.csv features (left) and unique_m.csv features (right) before feature reduction. Most train.csv features exhibit VIF values far exceeding 10.

VIF After Feature Selection

By retaining only one representative aggregation per physical property group (e.g., keeping wtd_range_atomic_mass while dropping the other 9 aggregations of atomic mass), the predictor set was reduced to just 10 features. The VIF values dropped dramatically to acceptable levels:

Feature	VIF
number_of_elements	2.05
wtd_range_atomic_mass	3.95
wtd_range_fie	1.35
wtd_range_atomic_radius	2.20
wtd_range_Density	3.49
mean_ElectronAffinity	1.10
mean_FusionHeat	1.45
mean_ThermalConductivity	1.28
wtd_std_Valence	1.44
critical_temp	2.17

All VIF values are now below 5, confirming that this reduced feature set is free of problematic multicollinearity.

Figure 9: After reducing to one aggregation per group, all VIF values fall well below 5, eliminating multicollinearity concerns.

However, since the purpose of this study is to evaluate regularization methods (which are specifically designed to handle multicollinearity), we proceeded to modeling using all available predictors rather than this reduced set.

Modeling Strategy

Data Preparation

1. Feature combination: The scaled train.csv features (81 predictors) were merged with the unscaled elemental composition features from unique_m.csv (86 element columns), producing a combined predictor matrix of 167 features.
2. Scaling: StandardScaler was applied to the train.csv features to standardize them (zero mean, unit variance). The elemental composition features were left unscaled, as they represent raw counts/proportions.
3. Train-test split: An 80/20 split was used (random_state=1234) with shuffling.
4. Cross-validation: 5-fold cross-validation (KFold, shuffle=True, random_state=1234) was used for hyperparameter tuning.
5. Target transformation: For Ridge Regression, the target variable was Box-Cox transformed to correct the right skew. For Lasso Regression, the raw target was used directly.

Figure 10: Side-by-side histograms of the training target before (left) and after (right) Box-Cox transformation, showing the skew correction applied for Ridge regression.

Understanding Regularization

In ordinary least squares (OLS) regression, the model minimizes only the sum of squared residuals. When predictors are highly correlated (as in our case with 167 features), OLS coefficients become unstable and can take on very large values.

Regularization adds a penalty term to the loss function that discourages large coefficients:

• Ridge (L2): Adds a penalty proportional to the sum of squared coefficients ((\alpha \sum \beta_j^2)). This shrinks coefficients toward zero but rarely sets any exactly to zero.

• Lasso (L1): Adds a penalty proportional to the sum of absolute coefficients ((\alpha \sum |\beta_j|)). This can shrink some coefficients exactly to zero, effectively performing automatic feature selection.

The regularization strength (\alpha) controls the trade-off between fitting the data and keeping coefficients small. Both RidgeCV and LassoCV select the optimal (\alpha) via cross-validation.

Results

Ridge Regression (L2 Regularization)

Ridge regression was trained on the Box-Cox-transformed target using RidgeCV with alpha values searched over np.logspace(-5, 5, 11).

Performance:

Metric	Value
Best CV Score (neg. MSE)	-0.1038
Test-set R²	0.915
Test-set MSE	103.53

The model explains approximately 91.5% of the variance in the critical temperature on the held-out test set, which is a strong result.

Top 10 Most Important Features (by absolute coefficient):

| Rank | Feature | Coefficient | |Coefficient| | |---|---|---|---| | 1 | critical_temp (from df2) | 0.707 | 0.707 | | 2 | number_of_elements | 0.152 | 0.152 | | 3 | wtd_range_Density | -0.086 | 0.086 | | 4 | Ce (Cerium) | -0.066 | 0.066 | | 5 | In (Indium) | -0.061 | 0.061 | | 6 | Ag (Silver) | -0.060 | 0.060 | | 7 | Fe (Iron) | 0.057 | 0.057 | | 8 | Ca (Calcium) | -0.053 | 0.053 | | 9 | Zn (Zinc) | -0.050 | 0.050 | | 10 | mean_ElectronAffinity | -0.047 | 0.047 |

Key observations:

• The critical_temp feature from the elemental composition dataset (unique_m.csv) dominates with the largest coefficient, suggesting data leakage (discussed further below).
• number_of_elements is the second most influential feature, indicating that the complexity of the compound plays a significant role.
• Several individual elements (Ce, In, Ag, Fe, Ca, Zn) appear in the top 10, suggesting that specific elemental presence strongly influences superconducting behavior.
• Ridge regression retains all 167 features with non-zero coefficients, distributing the predictive weight broadly.

Lasso Regression (L1 Regularization)

Lasso regression was trained on the raw (untransformed) target using LassoCV with automatic alpha selection.

Performance:

Metric	Value
Optimal Alpha	0.183
Test-set R²	1.000
Test-set MSE	0.040

The model achieves a near-perfect R² of 1.0 — an unusually high value that warrants careful scrutiny.

Top 10 Features (by absolute coefficient):

Rank	Feature	Coefficient
1	critical_temp (from df2)	34.023
2	O (Oxygen)	0.022
3–10	All other features	0.000

Key observations:

• Lasso’s L1 penalty shrinks all but 2 features to exactly zero, performing aggressive feature selection.
• The critical_temp feature from unique_m.csv absorbs nearly all the predictive power with a coefficient of ~34.
• Only O (Oxygen) retains a tiny non-zero coefficient (0.022), likely reflecting the prevalence of oxide-based superconductors (cuprates) in the dataset.

Lasso Regularization Path

The Lasso path plot illustrates how coefficients evolve as the regularization parameter (\alpha) decreases (less regularization). As (\alpha) shrinks, more features enter the model with non-zero coefficients. Conversely, at higher (\alpha) values, most coefficients are driven to zero. The accompanying RMSE curve shows that model error initially decreases rapidly as key features enter the model, then plateaus once the most important predictors have been included.

Discussion: The Data Leakage Issue

A critical finding in this analysis is the presence of data leakage. Both the train.csv and unique_m.csv datasets contain a critical_temp column. When these datasets are merged for modeling, the critical_temp from unique_m.csv becomes one of the predictor variables — but it is essentially the same variable we are trying to predict.

This explains:

• Why Lasso achieves R² ≈ 1.0: The model simply learns that critical_temp (predictor) ≈ critical_temp (target), with a coefficient of ~34 that rescales the Box-Cox / raw values.
• Why Lasso zeroes out all other features: When the target is directly available as a predictor, no other feature adds meaningful information.
• Why Ridge also ranks critical_temp as the top feature: Even though Ridge retains all features, the leaked variable dominates.

In a real-world application, one should drop the critical_temp column from unique_m.csv before merging datasets to avoid this leakage. The Ridge results without leakage would likely show a lower R² but would represent genuine predictive performance.

Key Takeaways

1. Multicollinearity is pervasive in engineered feature sets. When the same physical property is summarized through 10 different statistical aggregations, these summaries are inevitably correlated. VIF analysis is a simple and effective diagnostic tool.

2. Regularization handles multicollinearity gracefully. Rather than manually selecting one feature per group (which the VIF analysis demonstrated could reduce 81 features to 10 with acceptable VIFs), Ridge and Lasso automatically manage correlated predictors through their penalty terms.

3. Ridge vs. Lasso serve different purposes:

   • Ridge is ideal when you believe all features contribute and you want stable coefficient estimates. It shrinks coefficients but retains all of them.
   • Lasso is ideal for feature selection. It produces sparse models by driving irrelevant coefficients exactly to zero, making results more interpretable.

4. Data leakage is a common pitfall in machine learning pipelines. When merging multiple data sources, always verify that no form of the target variable appears among the predictors. A model with suspiciously perfect performance (R² ≈ 1.0) should raise a red flag.

5. Box-Cox transformation can be a valuable preprocessing step when the target variable is right-skewed. It helps linear models meet the normality assumption, potentially improving both stability and performance.

Conclusion

This case study demonstrates the practical application of L1 and L2 regularization to a real-world materials science problem: predicting the critical temperature of superconductors. Through exploratory data analysis, VIF diagnostics, and model comparison, we showed that regularized regression methods can effectively handle high-dimensional, multicollinear data.

The Ridge model achieved a strong R² of 0.915 on the test set when using a Box-Cox-transformed target, while the Lasso model revealed, through its aggressive feature selection, that the merged dataset contained data leakage via a duplicated target variable. This finding underscores the importance of careful data validation before model training.

For future work, removing the leaked feature and comparing model performance on the clean dataset would provide a more honest assessment of predictive accuracy. Additionally, exploring non-linear models (e.g., Random Forests, Gradient Boosting) may capture complex relationships between elemental properties and superconducting behavior that linear models miss.

---

This analysis was conducted using Python with scikit-learn, pandas, NumPy, seaborn, matplotlib, and statsmodels.