• The Client Will Subscribe to a Term Deposit or Not?
  • Source
  • Prerequisites
    • Tools and Environment
• Exploratory Data Analysis
  • Understanding the Business Problem
  • Data Overview
  • Numerical Features Description
  • Categorical Features Description
  • Correlation Matrix
  • Count of Term Deposits Subscriptions
  • Boxplot of Numerical Features vs Term Deposit Subscription
  • Count Plot of Categorical Features vs Term Deposit Subscription
• Data Pre-Processing
• Train and Test Split
• Model Building
  • Model Performance
• Conclusion
  • Feature Importance
• Recommendations
• Next Steps

This project investigates a dataset related to direct marketing campaigns of a Portuguese banking institution. Often, multiple contacts with the same client were required to assess if the product (bank term deposit) would be ('yes') or not ('no') subscribed. The classification goal is to predict if the client will subscribe (yes/no) to a term deposit (variable y).

[Moro et al., 2014] S. Moro, P. Cortez, and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014.

Jupyter Notebook: Preferably via Anaconda-Navigator or any IDE supporting Jupyter Notebooks.

Python Version: 3.11.5

Essential Libraries

matplotlib 3.7.2
seaborn 0.12.2
pandas 2.0.3
numpy 1.21.5
sklearn 1.0.2

The primary objective is to predict whether a client will subscribe to a term deposit. This involves analyzing the effectiveness of marketing campaigns and identifying key factors influencing the subscription decision.

The dataset is available in two versions:

1. bank-additional-full.csv: Contains all examples (41,188 rows) and 20 inputs, ordered by date (from May 2008 to November 2010).
2. bank-additional.csv: A smaller dataset (4,119 rows) used for testing more computationally demanding machine learning algorithms (e.g., SVM).

For this project, the smaller dataset was used to reduce computational complexity and time.

Descriptive statistics for numerical features:

Descriptive statistics for categorical features:

• Duration vs Term Deposit Subscription: Clients who subscribed to term deposits had a higher median duration compared to those who did not.
• Consumer Price Index vs Term Deposit Subscription: Subscriptions to term deposits looks good when consumer price index is low.
• Euro Interbank Offered Rate 3 Months vs Term Deposit Subscription: Subscriptions to term deposits are higher when the Euribor 3-month rate is low.

1. Imputed missing values for columns 'job', 'marital', 'education', 'default', 'housing', 'loan' using SimpleImputer with the most frequent strategy.
2. Excluded 'duration' for predictive modeling as it is only known after the call is performed.
3. Converted y to integer type (0 for 'no' and 1 for 'yes').
4. Applied one-hot encoding for categorical columns and standard scaling for numerical columns as part of the pipeline to avoid data leakage.

• Split the data into training and testing sets with a test size of 30%.
• Used random_state set to 25 for reproducibility
• Employed stratify=y to maintain class distribution in training and testing sets.

Given the imbalance in the dataset (11% 'yes' class), just accuracy is not a reliable metric. Even dummy model is giving 89.03% of accuracy. The focus is on recall and precision, aiming for a higher F1 score to balance false positives and correctly identified subscriptions.

Initial models with default hyperparameters and tuned models using GridSearchCV with 5-fold cross-validation and F1 score as the metric were tested.

Out of all the models evaluated, Logistic Regression with default hyperparameters proved to be the best, achieving the highest F1 score of 0.375691.

Below are the performance metrics presented in bar plots for all the models:

Training Time

• SVC has the longest training time at 1.560230 seconds, making it the least efficient in terms of training speed.
• Logistic Regression follows with a training time of around 0.52 seconds, which is relatively moderate.
• KNN and Decision Tree models are much faster, with training times of 0.020142 and 0.040505 seconds, respectively, indicating their efficiency in training.

Accuracy

• Decision Tree shows the highest training accuracy at 0.999306 but significantly drops in test accuracy to 0.828479, indicating overfitting.
• Logistic Regression and SVC have similar training and test accuracies (Logistic: 0.902879 train, 0.908576 test; SVC: 0.910163 train, 0.902913 test), suggesting good generalization.
• KNN has a slightly lower test accuracy (0.889968) compared to its training accuracy (0.912591), indicating a moderate generalization capability.

F1 Score

• Decision Tree has the highest training F1 score (0.996825) but a lower test F1 score (0.278912), indicating overfitting.
• Logistic Regression maintains a reasonable balance with training (0.351852) and test (0.375691) F1 scores, suggesting good generalization.
• KNN and SVC have relatively lower test F1 scores (KNN: 0.291667, SVC: 0.285714), with KNN showing a significant drop from its training F1 score (0.454545).

ROC Curve

The ROC curves visually indicate that Logistic Regression, Logistic Regression with F1, and KNN with F1 have high AUC values.

However, when calculating the ROC AUC scores, both Logistic Regression with default hyperparameters and Logistic Grid achieved the highest scores of 0.757628 and 0.758126, respectively. Although these two models are very close in performance, Logistic Regression with default hyperparameters has a higher F1 score.

ROC AUC Scores

The best performing model is Logistic Regression with default hyperparameters, achieving an F1 score of 0.38 and an accuracy of 90.87%. This model can effectively predict whether a client will subscribe to a term deposit.

Logistic Regression with default hyperparameters confusion matrix

1. Focus on Key Months: Marketing campaigns should be intensified in March, December, October, November, and September as these months show a higher likelihood of term deposits.
2. Leverage Previous Campaign Outcomes: Clients with 'success' or 'nonexistent' outcomes in previous campaigns are more likely to subscribe.
3. Optimize Contact Type: Prioritize cellular contacts, which have a higher success rate.
4. Consider Economic Indicators: Monitor the consumer price index and employment variation rate as they are significant predictors.

1. Collect more data to enhance model performance.
2. Incorporate additional features to capture more predictive power.
3. Experiment with advanced algorithms and ensemble methods like Random Forest, Gradient Boosting, etc.
4. Implement techniques to handle class imbalance more effectively.
5. Assess the model's performance on the full dataset.
6. Automate the data pipeline and model training process for real-time predictions.
7. Implement the model in a real-world scenario and monitor its performance.
8. Keep the model updated with new data and retrain it periodically.
9. Conduct a cost-benefit analysis to determine the optimal threshold for the model.