Four end-to-end production-grade projects at the intersection of quantitative finance, machine learning, and regulatory compliance. Built during the FinTech module at Politecnico di Milano (A.Y. 2025–2026).
Contributors: Matteo Campagnoli · Luigi Di Gregorio · Riccardo Girgenti · Giacomo Kirn · Francesco Ligorio
This repository covers the full ML-in-finance stack — from raw client data to deployable decision engines — across four distinct problem domains:
| # | Problem Domain | Core Challenge |
|---|---|---|
| 1 | Client Segmentation | Mixed-type distance on heterogeneous demographic data |
| 2 | Recommendation & Next Best Action | Multi-label classification + MiFID II–compliant suitability filtering |
| 3 | Portfolio Replication | Reverse-engineering a black-box hedge fund index with liquid futures |
| 4 | Early Warning System | Unsupervised anomaly detection on financial time series |
Stack: Python 3.9+ · PyTorch · LightGBM · XGBoost · Scikit-Learn · Optuna · SHAP · hmmlearn · filterpy
Goal: Move beyond AUM-bracket segmentation and build data-driven client personas that a wealth manager can act on.
Mixed-Data Distance Design Raw client features span nominal variables (Job, Area, CitySize), ordinal variables (InvestmentExperience), and continuous financial metrics (Wealth, Income, Age). Standard Euclidean distance is meaningless here. We implement Gower Dissimilarity to construct a unified distance matrix: nominal features contribute binary mismatch indicators, ordinal and continuous features contribute range-normalized absolute differences.
Prototype-Based Clustering (K-Medoids) We run K-Medoids directly on the precomputed Gower matrix. Unlike K-Means, medoids are real client profiles — each cluster center is an actual person, making business interpretation unambiguous. Optimal k is selected via silhouette analysis and gap statistics.
LVQ1 Refinement Cluster prototypes are refined post-hoc with Learning Vector Quantization (LVQ1), an online competitive learning algorithm that adjusts prototype positions based on correct vs. incorrect classification signals, sharpening cluster boundaries.
Topological Data Analysis — Mapper Algorithm The Mapper algorithm compresses the full client manifold into a relational graph without fixing k in advance. Nodes represent client subgroups; edges encode overlap. This reveals continuous gradients (income transitions, life-stage trajectories) that hard-boundary clustering inherently misses.
Domain-Driven Feature Engineering
- Wealth/Income ratio → saving capacity proxy
- Log-scale normalization → compress heavy-tailed wealth distributions
- Age × FamilyMembers → life-stage burden index
- FinancialEducation × RiskPropensity → sophistication proxy
Goal: Predict each client's investment needs and route them to MiFID II–compliant product recommendations.
Multi-Label Classification Pipeline
Two correlated binary targets — Income Investment propensity and Accumulation Investment propensity — are modeled jointly via MultiOutputClassifier(LightGBM), capturing label correlation rather than treating targets independently. Benchmarked against Logistic Regression, Naive Bayes, KNN, SVM, Random Forest, XGBoost, and Gradient Boosting.
Bayesian Hyperparameter Optimization (Optuna / TPE) Grid search is replaced with Tree-structured Parzen Estimator (TPE) optimization via Optuna. The sampler builds probabilistic models of the objective function and focuses trials on promising hyperparameter regions, reaching competitive performance in a fraction of brute-force evaluations. Extended to XGBoost and Random Forest to check leaderboard stability.
Stacking Ensemble A two-level meta-learner (Wolpert, 1992) trains a regularized Logistic Regression on out-of-fold predictions from four base learners: LightGBM, XGBoost, Logistic Regression, and a PyTorch MLP. Base models capture different aspects of the feature space; the meta-learner learns optimal combination.
PyTorch MLP with Architecture Sweep
Manual neural baseline with BatchNorm, Dropout, Adam optimizer, ReduceLROnPlateau scheduling, and early stopping. Width/depth sensitivity analyzed across architectures.
Probability Calibration (Platt Scaling) Raw model scores are converted to calibrated probabilities via Platt Scaling (isotonic and sigmoid variants). Required for the NBA engine to apply meaningful probability thresholds, not just rank ordering.
Bayesian Model Averaging (BMA) Frequentist LightGBM probability scores are combined with Bayesian Beta-Binomial posteriors:
p̂(x) = w · p_LGBM(x) + (1-w) · p_Bayes(x)
The Bayesian component anchors predictions toward empirical base rates, mitigating advisor commission bias embedded in historical training labels. Prior parameters are set via Empirical Bayes from training-set prevalence.
Explainable AI (XAI)
- Global SHAP: feature importance and interaction effects across the full client population
- Local SHAP (waterfall plots): per-client decision decomposition for advisor-facing explanations
- Partial Dependence Plots: marginal effect of each feature, holding others at their expected value
Collaborative Filtering (SVD & Autoencoder) Exploratory deployment template: SVD and a non-linear Autoencoder reconstruct a client-product interaction matrix. When real purchase history is available, these serve as latent-factor recommendation engines.
Fairness Audit (MiFID II / RIS) Three fairness metrics computed on the held-out test set:
- Demographic Parity: equal positive prediction rates across gender groups
- Equal Opportunity: equal True Positive Rates across groups
- Predictive Parity: equal precision across groups
5-Stage NBA Engine
- Calibrated probability scoring per client
- Product catalog ingestion
- Hard suitability filter:
|client_risk − product_risk| ≤ 0.25 - Multi-attribute preference ranking
- Top-k recommendation with explainability output
Goal: Reverse-engineer a non-investable black-box portfolio (Monster Index: 50% HFRXGL hedge fund index, 25% MXWO, 25% LEGATRUU) using only 11 liquid futures contracts.
Return Unsmoothing — Geltner Filter HFRXGL exhibits significant lag-1 autocorrelation: hedge funds hold illiquid assets and report smoothed NAVs. The Geltner (1993) model corrects this:
r_obs(t) = α · r_true(t) + (1-α) · r_obs(t-1)
Inverting recovers unsmoothed true returns, exposing hidden volatility and enabling proper hedge sizing.
Static Linear Benchmarks Full benchmark suite on a fixed train/test split (cutoff: 2018-01-01): OLS, Ridge, Lasso, Elastic Net. Lasso enforces futures pre-selection via coefficient sparsity; Ridge controls leverage; Elastic Net interpolates.
Rolling Window Models (Walk-Forward) At each step t, the model is re-estimated on a fixed-width rolling window of past weeks, then used to predict the next week only. Weights adapt dynamically, removing the constant-weight assumption that breaks down under regime shifts. Benchmarks: Rolling OLS, Rolling Elastic Net with VaR constraint, rebalancing frequency analysis.
Hyperparameter Sensitivity Analysis Full α × l1_ratio grid heatmap for Elastic Net. A flat valley (wide low-TE region) is preferred over a sharp minimum — identifies robustness of the optimal hyperparameter choice out-of-sample.
Hidden Markov Model (HMM) Regime Detection A 3-state Gaussian HMM fits latent regimes on Monster Index returns:
| State | Interpretation | Characteristics |
|---|---|---|
| 0 | Neutral | Moderate returns, moderate vol |
| 1 | Bull / Low Vol | Positive drift, compressed vol |
| 2 | Bear / Crisis | Negative drift, elevated vol |
Replication models are evaluated conditionally on regime, exposing which models hold up under stress vs. which only perform in benign markets.
Kalman Filter (State-Space Tracking) Portfolio weights are modeled as unobservable latent states evolving via a random walk:
x_t = A x_{t-1} + Bu_t (state equation — weights as latent states)
y_t = C_t x_t + D var_t (observation equation — realized returns)
The Kalman Filter recursively updates weight estimates as new data arrives, with no fixed window assumption. Naturally propagates estimation uncertainty.
WeightNet — Neural Meta-Model A feedforward neural network generates portfolio weights directly:
w_t = f_θ(z_t)
where z_t is a feature vector of rolling statistics (means, volatilities, correlations) computed from the futures return history. Trained end-to-end to minimize tracking error.
Transaction Cost Analysis Net returns account for 5bps per trade:
net_return_t = gross_return_t − 0.0005 × Σ|w_i,t − w_i,t-1|
Turnover-adjusted performance evaluated across all models. Rebalancing frequency directly optimized.
Evaluation Metrics Tracking Error (TE), Information Ratio, regime-conditional TE breakdown, Sharpe Ratio, Maximum Drawdown, Cumulative Return.
Goal: Detect abnormal market conditions in financial time series before they manifest as portfolio losses.
Time-Series Stationarization
- Log-returns for price/level series
- First differences for rate series (%, weekly change)
- Strict chronological train/CV/test split (60/20/20) — no shuffle, no look-ahead bias
Multivariate Gaussian (MVG) Baseline μ and Σ estimated on normal-regime data only. Anomaly score = log-likelihood:
Anomaly ⟺ log p(x) < ε
Threshold ε is selected by maximizing F1 on the validation set.
Isolation Forest Non-parametric spatial partitioning: anomalies are observations that require fewer random binary splits to isolate. No distributional assumption, robust to high-dimensional feature spaces.
Deterministic Autoencoder (AE) Encoder-decoder architecture trained on normal data only; reconstruction MSE serves as anomaly score. High MSE → the observation lies off the normal manifold. Reconstruction error tracked over time and visualized in PCA space.
Variational Autoencoder (VAE) Probabilistic extension: the encoder outputs a distribution over latent codes (μ, σ), sampled via the reparameterization trick. Trained with ELBO loss (reconstruction + KL divergence). The regularized latent space yields smoother anomaly scores and explicit uncertainty quantification.
AUC-Weighted Ensemble Individual model anomaly scores are combined via AUC-weighted averaging, giving higher weight to models that historically discriminate better on the validation regime. Non-linear score combination outperforms equal-weight averaging.
Financial Backtesting Detection signals validated against MSCI USA Index via lagged trading signals. Evaluated on: Sharpe Ratio, Maximum Drawdown, Cumulative Return, signal hit rate per regime.
| Concern | Implementation |
|---|---|
| Temporal integrity | Chronological splits everywhere; no random shuffle on time series |
| Leakage prevention | Features computed strictly on training window; rolling re-estimation at each step |
| Imbalanced labels | AUC as primary metric; threshold tuning on validation set |
| Heavy-tailed distributions | Log-scaling for wealth/income; robust statistics for rolling moments |
| Leverage control | VaR ceiling constraints in Elastic Net; Ridge regularization |
| Regulatory alignment | MiFID II suitability filters; IDD/RIS fairness audits |
.
├── customer-segmentation/
│ ├── main.ipynb # Gower matrix, K-Medoids, LVQ1, Mapper
│ └── extras.ipynb # Extended topology and sensitivity analysis
│
├── recommendation-nba/
│ ├── main_Group2.ipynb # Full modeling pipeline + NBA engine
│ └── extras_Group2.ipynb # Stacking, BMA, fairness audit, collaborative filtering
│
├── portfolio-replication/
│ └── Business_Case_3_Group2.ipynb # Static/rolling models, HMM, Kalman, WeightNet
│
├── early-warning-system/
│ └── BC4_Group2.ipynb # MVG, Isolation Forest, AE, VAE, ensemble, backtesting
│
└── README.md
pip install torch lightgbm xgboost scikit-learn optuna shap hmmlearn filterpy numpy pandas matplotlib seabornPython 3.9+ · Tested on CPU; GPU optional for PyTorch components.
Politecnico di Milano — FinTech Master Module, A.Y. 2025–2026