Aml Midsem
Uploaded by
VANSH SHAHAml Midsem
Uploaded by
VANSH SHAHNumericals topics:-
● Decision tree(Information gain, gini index, gain ratio, CART):
Gini index:https://sefiks.com/2018/08/27/a-step-by-step-cart-decision-tree-example/
Binning:
● Z-Transformation:
● Encoding:
● Normalization or Standardization::
R
● egression:
1. Linear
● Performance matrix(Accuracy, Precision, Recall, F1-Score):
○ ROC curve/Confusion matrix:
● least square regression:
tep 1:Denote the independent variable values asxi and the dependent ones as yi.
S
Step 2:Calculate the average values of xi and yi as X and Y.
Step 3:Presume the equation of the line of best fitas y = mx + c, where m is the slope of the line and c
represents the intercept of the line on the Y-axis.
Step 4:The slope m can be calculated from the followingformula:
m = [Σ (X – xi)×(Y – yi)] / Σ(X – xi)2
Step 5:The intercept c is calculated from the followingformula:
c = Y – mX
Thus, we obtain the line of best fit as y = mx + c, where values of m and c can be calculated from the
formulae defined above.
Problem 1: Find the line of best fit for the following data points using the least squares method: (x,y)
= (1,3), (2,4), (4,8), (6,10), (8,15).
Here, we have x as the independent variable and y as the dependent variable. First, we calculate the
means of x and y values denoted by X and Y respectively.
X = (1+2+4+6+8)/5 = 4.2
Y = (3+4+8+10+15)/5 = 8
he slope of the line of best fit can be calculated from the formula as follows:
T
m = (Σ (X – xi)*(Y – yi)) /Σ(X – xi)2
m = 55/32.8 = 1.68 (rounded upto 2 decimal places)
Now, the intercept will be calculated from the formula as follows:
c = Y – mX
c = 8 – 1.68*4.2 = 0.94
Thus, the equation of the line of best fit becomes, y = 1.68x + 0.94.
Unit 1
1. Data Preparation
ata preparation is the process of making raw data ready for after processing and analysis. The key
D
methods are to collect, clean, and label raw data in a format suitable for machine learning (ML)
algorithms, followed by data exploration and visualization. The process of cleaning and combining raw
data before using it for machine learning and business analysis is known as data preparation, or
sometimes “pre-processing.” Data preparation is like cleaning and organizing your data so that it’s
ready to be used for analysis or machine learning. Think of it like getting ingredients ready before
cooking. You need everything clean, chopped, and in the right place before you start.
Key Steps in Data Preparation:
1. D ata Cleaning: This means fixing mistakes in the data.For example, correcting spelling errors
or filling in missing information.
2. Feature Selection: Choosing the most important piecesof data that will help your model make
good predictions. Not every piece of data is useful.
3. Data Transformation: Changing the data into a formatthat the machine learning model can
understand. For example, turning dates into numbers or categories.
4. F eature Engineering: Creating new information from the existing data to improve the model.
For example, if you have a “date,” you could create a new feature like “day of the week.”
5. Dimensionality Reduction: Reducing the number of variableswhile keeping the important
information, so the model doesn't get confused by too many details.
Steps to Follow:
1. U nderstand the Problem: Know what you are trying tosolve. It’s important to understand the
problem fully before working with the data.
2. Collect Data: Gather all the data from different sources.Make sure the data represents
different situations so that the model works for everyone.
3. Explore the Data: Look at the data to find any unusualor missing values and trends. This helps
you understand if anything needs fixing.
4. Clean and Validate the Data: Fix any problems likemissing values or incorrect information.
Clean data helps make better predictions.
5. Format the Data: Make sure the data is consistent(e.g., all prices have the same currency
symbol).
6. Improve Data Quality: Combine similar columns or piecesof data to ensure everything makes
sense. For example, merging "First Name" and "Last Name" into one field.
7. Feature Engineering: Create new features from yourdata to improve the model. For example,
you can split a date into "day," "month," and "year" to provide more details.
8. Split the Data: Divide the data into two sets – onefor training the model and one for testing it.
Benefits of Good Data Preparation:
B
● etter Predictions: Clean and organized data makesthe model smarter and more accurate.
● Fewer Mistakes: Fixing errors in the data means fewerwrong predictions.
● Faster and Better Decisions: Well-prepared data makesit easier to make decisions based on
analysis.
● Save Time and Money: Clean data reduces the chanceof having to redo work later.
● Stronger Models: Well-prepared data helps create morereliable machine learning models.
2. Data Pre-processing:-
reprocessing stepsare thespecific tasksyou performon raw data to prepare it for analysis or model
P
training. These steps ensure the data is clean, consistent, and in the right format for the machine
learning algorithm to process effectively.
Importance of Data Pre-processing
ata preprocessing is a crucial step in any data analysis or machine learning pipeline. Here are key
D
reasons why it's important:
1. Improves Data Quality
H
● andles missing values, outliers, and inconsistencies.
● Ensures data accuracy and reliability.
● Helps eliminate noise that can negatively impact model performance.
2. Enhances Model Performance
S
● cales and normalizes data, leading to faster and more accurate training.
● Reduces feature redundancy, making the model more efficient.
● Helps in feature selection, focusing on the most relevant data for the task.
3. Increases Model Generalization
P
● revents overfitting by cleaning and simplifying the dataset.
● Facilitates better decision-making through clearer patterns in data.
● Ensures the model works well with unseen data by removing biases.
4. Saves Time and Computational Resources
R
● educes the dimensionality of data, leading to faster computations.
● Helps avoid errors early in the pipeline, saving effort in later stages.
● Makes the model training and testing more efficient.
5. Essential for Proper Feature Engineering
A
● llows for the extraction and transformation of features that can improve predictive power.
● Facilitates the creation of new features that capture more information.
● Aligns data formats, enabling proper comparison and analysis.
6. Facilitates Data Compatibility
C
● onverts data into a format suitable for the chosen algorithm.
● Ensures uniformity, especially when dealing with multiple datasets.
● Make sure data is in a structured format, ready for further analysis.
7. Enhances Interpretability
H
● elps visualize trends and patterns for easier interpretation.
● Reduces data complexity, making it more accessible for decision-makers.
● Facilitates clear understanding for stakeholders by removing irrelevant information.
Examples of Preprocessing Steps:
H
● andling Missing Values: Deciding how to deal withincomplete data points.
● Scaling/Normalization: Ensuring that numerical valuesfall within a consistent range.
● Encoding Categorical Data: Converting non-numericdata into a format that a machine learning
algorithm can interpret.
● Feature Selection: Choosing the most important features(variables) in your dataset for model
training.
● Outlier Removal: Identifying and removing abnormaldata points that may skew the model's
performance.
● Data Transformation: Applying functions like logarithmsor square roots to transform data into
a more usable form.
● Data Augmentation: Expanding the dataset artificiallyby generating new data from the existing
data (mainly used in image or text data).
Preprocessing Techniques:
reprocessing techniquesare themethodsorapproachesused to accomplish the preprocessing
P
steps. While preprocessing steps tell youwhat needs to be done, preprocessing techniques explain
howyou can do it.
Examples of Preprocessing Techniques:
2.1 Imputation: Filling missing values using methodslike:
1. Mean/Median/Mode Imputation
his technique fills missing values by replacing them with the mean, median, or mode
T
of the available data.
M
● ean: Good for numerical data without extreme outliers.
● Median: Good for skewed data or data with outliers.
● Mode: Best for categorical data (most frequent value).
xample: You have a dataset of house prices with missing values in the “Number of
E
Rooms” column.
House ID Number of Rooms Price
1 3 200K
2 4 250K
3 NaN 230K
4 5 300K
Steps:
Identify the missing value in House ID 3.
Calculate the mean of non-missing values (3+4+5)/3 = 4.
Replace the missing value with the mean, so House ID 3 now has 4 rooms.
House ID Number of Rooms Price
1 3 200K
2 4 250K
3 4 230K
4 5 300K
2. K-Nearest Neighbors (KNN) Imputation
NN Imputation estimates missing values by finding the “k” closest (most similar) rows
K
in the dataset and using their values to fill in the missing data.
xample: You have a dataset of patients where the “Weight” column has missing
E
values.
Patient ID Height (cm) Weight (kg) Age
1 170 70 25
2 165 65 22
3 180 NaN 30
4 175 80 28
5 160 55 20
Steps:
Identify the missing value for Patient 3 in the "Weight" column.
Choose 3 neighbors (k=3) based on similarity in height and age.
ind the nearest 3 patients and average their weights (Patient 1: 70 kg, Patient 2: 65 kg,
F
Patient 4: 80 kg).
Impute the missing weight for Patient 3 with the average: (70+65+80)/3 = 71.67 kg.
Patient ID Height (cm) Weight (kg) Age
1 170 70 25
2 165 65 22
3 180 71.67 30
4 175 80 28
5 160 55 20
3. Regression Imputation
egression Imputation predicts missing values using a regression model based on
R
other available features.
xample: You have a dataset of cars where the “Engine Size” is missing for some
E
entries.
Car ID Horsepower Engine Size (L) Fuel Efficiency (MPG)
1 150 2.0 30
2 200 NaN 25
3 250 3.5 20
4 180 2.5 27
5 220 3.0 22
Steps:
Identify the missing engine size for Car ID 2.
Use features like “Horsepower” and “Fuel Efficiency” to train a linear regression model.
Fit the model with the data of other cars.
Example model: Engine Size = a × Horsepower + b × Fuel Efficiency + c.
Predict the missing value for Car ID 2. Assume the predicted value is 2.8L.
Car ID Horsepower Engine Size (L) Fuel Efficiency (MPG)
1 150 2.0 30
2 200 2.8 25
3 250 3.5 20
4 180 2.5 27
5 220 3.0 22
2.2 Scaling/Normalization: Adjusting the range ofdata points using techniques like:
1. Min-Max Scaling: Scaling values to a fixed range,usually [0, 1].
Concept:
● M in-Max Scaling, often referred to asNormalization,rescales the data to fit within a specific
range, typically between0 and 1.
● This scaling method adjusts the values to a common scale, ensuring that no specific feature
dominates simply due to larger numeric values.
Steps Involved:
1. Identify the range of the feature:
○ Determine the minimum (Xmin) and maximum (Xmax) values for the feature that you
wish to scale.
2. Calculate the new values:
○ For each value X, subtract the minimum value Xmin from it and then divide the result by
the difference between Xmax and Xmin. This rescales the data to the 0-1 range.
3. Apply the transformation:
○ Apply the scaling formula to each value in the dataset to rescale it between 0 and 1 (or
another desired range).
4. Advantages:
○ Simple to implement and interpret.
○ Works well for algorithms likeneural networksandk-nearest neighbors (k-NN), where
the range of input values can affect model performance.
5. Limitations:
○ Sensitive to outliers: If there are outliers, theywill distort the scale because the
transformation depends on the range (minimum and maximum), which will stretch the
scale for normal data points.
. Z-Score Normalization (Standardization): Rescalingdata to have a mean of 0 and a standard
2
deviation of 1.
Concept:
● Z -Score Standardization(or simplyStandardization) transforms the data so that it has a mean
of 0 and a standard deviation of 1.
● This is useful for datasets where the features follow a normal distribution, or for algorithms
that assume data is normally distributed (e.g.,linearregression,logistic regression, and
k-means).
Steps Involved:
1. Calculate the mean (μ) and standard deviation (σ):
○ Determine the mean and standard deviation for the feature you wish to standardize. The
mean represents the average value of the feature, while the standard deviation indicates
how much the values deviate from the mean.
2. Subtract the mean from each value:
○ For each data point X, subtract the mean (μ). This will center the data around 0, so that
values above the mean will be positive and those below the mean will be negative.
3. Divide by the standard deviation:
○ To normalize the spread of values, divide the result by the standard deviation (σ). This
scales the data so that most of the values fall within a range of -3 to +3, assuming a
normal distribution.
4. Advantages:
○ It centers data around 0, which is helpful for algorithms likePCAor any model relying
on normally distributed data.
○ Ensures that each feature contributes equally to the result, regardless of its original
scale.
5. Limitations:
○ Sensitive to outliers: Since the mean and standarddeviation are influenced by extreme
values, outliers can distort the standardization and reduce its effectiveness for most
data points.
2.3Encoding Categorical Data:
1. One-Hot Encoding: Representing categorical variablesas binary vectors.
Example: Let’s consider a dataset with a categorical feature called Fruit:
ID Fruit
1 Apple
2 Banana
3 Orange
4 Apple
5 Grape
Steps:
Identify Categorical Variables:
The Fruit column is identified as categorical.
Create Binary Columns:
Create new binary columns for each unique fruit:
Fruit_Apple, Fruit_Banana, Fruit_Orange, Fruit_Grape
Assign Binary Values:
For each row, assign values based on the original Fruit value:
ID Fruit Fruit_Apple Fruit_Banana Fruit_Orange Fruit_Grape
1 Apple 1 0 0 0
2 Banana 0 1 0 0
3 Orange 0 0 1 0
4 Apple 1 0 0 0
5 Grape 0 0 0 1
Drop Original Categorical Column:
Remove the original Fruit column:
ID Fruit_Apple Fruit_Banana Fruit_Orange Fruit_Grape
1 1 0 0 0
2 0 1 0 0
3 0 0 1 0
4 1 0 0 0
5 0 0 0 1
2. Label Encoding: Converting categorical labels intonumeric values.
Example: Consider a dataset with a categorical feature called Education Level:
ID Education Level
1 High School
2 Bachelor's
3 Master's
4 PhD
5 Bachelor's
Steps:
Identify Categorical Variables:
The Education Level column is identified as ordinal.
Assign Integer Values:
Assign unique integers to each category based on their level of education:
High School = 0
Bachelor's = 1
Master's = 2
PhD = 3
Replace Categories:
Replace the original Education Level values with their corresponding integers:
ID Education Level Education_Level_Encoded
1 High School 0
2 Bachelor's 1
3 Master's 2
4 PhD 3
5 Bachelor's 1
3. T
arget/Mean Encoding: Replacing categories with themean of the target variable for each
category.
xample: Now, consider a dataset with a categorical feature Department and a target
E
variable Salary:
ID Department Salary
1 HR 70,000
2 IT 90,000
3 HR 75,000
4 Finance 100,000
5 IT 95,000
Steps:
Identify Categorical Variable and Target Variable:
Identify Department as the categorical variable and Salary as the target variable.
Calculate Mean of Target for Each Category:
Calculate the mean salary for each department:
HR: (70,000 + 75,000) / 2 = 72,500
IT: (90,000 + 95,000) / 2 = 92,500
Finance: 100,000 (only one entry)
Replace Categories with Mean Values:
Replace the Department values with their corresponding mean salaries:
ID Department Salary Department_Encoded
1 HR 70,000 72,500
2 IT 90,000 92,500
3 HR 75,000 72,500
4 Finance 100,000 100,000
5 IT 95,000 92,500
Final Table:
ID Department_Encoded Salary
1 72,500 70,000
2 92,500 90,000
3 72,500 75,000
4 100,000 100,000
5 92,500 95,000
2.4 Outlier Removal:
1. Z
-Score Method: The Z-Score method is a statisticaltechnique used to identify outliers by
measuring how far a data point deviates from the mean, in terms of standard deviations. A
Z-Score tells us how many standard deviations away a particular value is from the mean of the
dataset. If a value lies far enough from the mean, it may be considered an outlier.
Example: Exam Scores Dataset with an Outlier
Student Exam Score
A 75
B 85
D 60
G 200
2. Standard Deviation: It can be used to identify the outliers
Example: Handling Outliers Using Standard Deviation
Consider a dataset of exam scores:
Student Exam Score
A 75
B 85
C 90
D 200
E 80
2.5 Dimensionality Reduction:
1. P
rincipal Component Analysis (PCA): PCA is a statisticalmethod that transforms a dataset
into a new coordinate system, where the greatest variance by any projection lies on the first
coordinate (called the first principal component), the second greatest variance on the second
coordinate (the second principal component), and so on. This results in a lower-dimensional
representation of the data that captures its structure and variance.
teps:
S
https://medium.com/@dhanyahari07/principal-component-analysis-a-numerical-approa
ch-88a8296dc2dc
2. t -SNE (t-distributed Stochastic Neighbor Embedding):t-SNE is a popular dimensionality
reduction technique particularly suited for visualizing high-dimensional data. Unlike PCA, which
focuses on capturing variance, t-SNE aims to maintain the local structure of data, preserving
distances between points. This makes it highly effective for exploring clusters and patterns in
datasets, especially in applications like image and text data analysis.
2.6 Data Transformation:
L
○ og Transformation: Applying a logarithmic functionto reduce skewness in data.
○ Box-Cox Transformation: A family of transformationsto stabilize variance and
make data more normal-like.
Data Augmentation (for image data):
●
○ Rotation: Rotating images at random angles.
○ Flipping: Flipping images horizontally or vertically.
○ Zooming: Randomly zooming into parts of the image.
Detailed Differentiation
Aspect Preprocessing Steps Preprocessing Techniques
Definition he specifictasksinvolved in
T hemethodsorapproachesused to
T
preparing data. complete the steps.
Purpose efinewhat needs to be done E
D xplainhow the tasks are achievedwith
in data preprocessing. practical methods.
xample 1:
E Step: Handle missing data. echniques: Mean imputation, KNN
T
Missing Data imputation, regression imputation.
xample 2:
E tep: Normalize or
S echniques: Min-Max scaling, Z-score
T
Scaling Data standardize the data. standardization, robust scaling.
xample 3:
E tep: Encode categorical
S echniques: One-hot encoding, label
T
Encoding data. encoding, target encoding.
xample 4:
E Step: Remove outliers. Techniques: Z-score method, IQR method.
Outliers
rder in
O reprocessing steps come
P echniques are applied after steps are
T
Workflow first and outline the tasks to defined, offering specific solutions.
perform.
Relationship teps define thegoalsfor
S echniques are thetoolsto achieve those
T
preparing the data. goals.
Flexibility teps are more general and
S echniques are specific and technical, and
T
conceptual. there can be multiple techniques for each
step.
3. Data Encoding Techniques
ata encoding techniques are essential in machine learning as they allow categorical data, which is
D
often in non-numeric form, to be converted into numerical representations. This is crucial because
most machine learning algorithms require numerical input to function properly. Here’s a detailed
explanation of the most commonly used data encoding techniques:
1. One-Hot Encoding
efinition: One-Hot Encoding is a technique used to convert categorical variables into a format that
D
can be provided to machine learning algorithms. It creates binary columns for each category in the
original feature.
Methodology:
Identify the unique categories in the categorical variable.
●
● Create a new binary column for each unique category.
● F
or each observation, assign a value of 1 in the column corresponding to the category it
belongs to, and 0 in all other columns.
Example:
Consider a dataset with a categorical feature "Fruit":
Fruit
Apple
Orange
Banana
Apple
Banana
After One-Hot Encoding, the dataset would look like this:
Apple OrangeBanana
1 0 0
0 1 0
0 0 1
1 0 0
0 0 1
When to Use:
● B
est for nominal data (categories without intrinsic order) where you want to prevent algorithms
from assuming a natural ordering.
Advantages:
P
● revents misleading assumptions about ordinal relationships in categorical data.
● Works well with many machine learning algorithms, including linear regression and neural
networks.
Disadvantages:
● C an significantly increase dimensionality, leading to the "curse of dimensionality" if there are
many unique categories.
● Sparse data representation (many zeros) can increase computation time.
2. Label Encoding
efinition: Label Encoding is a technique that converts categorical values into integer values. It
D
assigns a unique integer to each category.
Methodology:
Identify unique categories in the categorical variable.
●
● Assign each category a unique integer value, usually starting from 0.
Example:
Consider a dataset with a categorical feature "Color":
Color
Red
Green
Blue
Green
Red
After Label Encoding, the dataset would look like this:
Color Label Encoded Value
Red 0
Green 1
Blue 2
Green 1
Red 0
When to Use:
● Best suited for ordinal data where there is a natural order among the categories.
Advantages:
S
● imple to implement and efficient in terms of storage.
● Works well with algorithms that can handle numerical values (e.g., tree-based models).
Disadvantages:
N
● ot suitable for nominal data, as it introduces a false sense of ordering.
● Algorithms may interpret encoded values as having a rank or relationship that doesn't exist.
3. Ordinal Encoding
efinition: Ordinal Encoding is similar to Label Encoding but specifically used for ordinal categorical
D
data, where categories have a meaningful order.
Methodology:
Identify unique ordered categories in the categorical variable.
●
● Assign integer values to each category based on their order.
Example:
Consider a dataset with a feature "Education Level":
Education Level
High School
Bachelor
Master
PhD
Bachelor
After Ordinal Encoding, the dataset would look like this:
Education Level Ordinal Encoded Value
High School 1
Bachelor 2
Master3
PhD 4
Bachelor 2
When to Use:
● Best for ordinal data where the order of categories is significant.
Advantages:
M
● aintains the natural order of categories.
● Efficient for algorithms that can interpret numerical values correctly.
Disadvantages:
U
● sing it for nominal data can introduce incorrect relationships.
● Assumes that the difference between ranks is meaningful, which may not always hold.
. What is Feature Engineering?
4
Feature engineering is the process of transforming raw data into features that are suitable for machine
learning models. In other words, it is the process of selecting, extracting, and transforming the most
relevant features from the available data to build more accurate and efficient machine learning
models.
he success of machine learning models heavily depends on the quality of the features used to train
T
them. Feature engineering involves a set of techniques that enable us to create new features by
combining or transforming the existing ones. These techniques help to highlight the most important
patterns and relationships in the data, which in turn helps the machine learning model to learn from
the data more effectively.
1. Importance of Feature Engineering
● M odel Performance:Well-engineered features can leadto better predictive performance by
capturing the underlying patterns in the data that models need to learn from.
● Dimensionality Reduction:Feature engineering techniquescan help reduce the number of
features while retaining essential information, improving computational efficiency and reducing
overfitting.
● Interpretability:Properly engineered features canmake the model more interpretable and allow
stakeholders to understand the factors influencing predictions.
● Domain Knowledge Utilization:Feature engineeringleverages domain knowledge, allowing the
creation of features that are more relevant to the problem domain.
Key Concepts in Feature Engineering
a. Feature Creation
reating new features from existing data can capture hidden relationships and patterns. This can be
C
done through:
● M athematical Operations:Combining features usingaddition, subtraction, multiplication, or
division. For example, calculating the "body mass index (BMI)" from weight and height features.
● Date and Time Features:Extracting day, month, year,day of the week, or hour from date-time
variables. For instance, categorizing seasons from dates.
● Textual Data Processing:Extracting features fromtext data, such as word counts, sentiment
scores, or keyword presence.
b. Feature Transformation
ransforming existing features can enhance their utility for modeling. Common transformation
T
techniques include:
● N ormalization and Standardization:Rescaling featuresto a specific range or making them
zero-mean and unit-variance to ensure comparability.
● Log Transformations:Applying logarithmic transformationsto skewed features to reduce
skewness and stabilize variance.
● Binning/Bucketing:Converting continuous variablesinto categorical bins (e.g., age groups).
c. Feature Selection
Identifying and selecting the most relevant features helps to improve model performance and reduce
complexity. Techniques include:
● F ilter Methods:Statistical tests to evaluate theimportance of features independently (e.g.,
chi-squared test, correlation coefficient).
● Wrapper Methods:Utilizing a predictive model to evaluatecombinations of features (e.g.,
recursive feature elimination).
● E
mbedded Methods:Feature selection algorithms integrated within model training (e.g., LASSO
regression).
5. Feature Engineering Process
a. Understand the Problem
egin by thoroughly understanding the problem domain, the goals of the analysis, and the type of data
B
available.
b. Data Exploration and Cleaning
● E xploratory Data Analysis (EDA):Investigate the data'sstructure, distributions, and
relationships among variables. Tools like histograms, scatter plots, and correlation matrices
can help.
● Data Cleaning:Handle missing values, outliers, anderrors in the dataset to prepare it for
feature engineering.
c. Feature Creation and Transformation
C
● reate new features using domain knowledge and insights gained from data exploration.
● Apply appropriate transformations to enhance feature effectiveness.
. Feature Selection
d
https://www.javatpoint.com/feature-selection-techniques-in-machine-learning
eature selectionis the process of identifying andselecting a subset of relevant features (variables,
F
predictors) from a larger set of features in a dataset. The primary goal is to improve the performance
of machine learning models by eliminating irrelevant or redundant data that does not contribute to the
predictive power of the model.
Importance of Feature Selection
1. I mproves Model Performance:By focusing on the mostrelevant features, models can achieve
higher accuracy and better generalization on unseen data.
2. Reduces Overfitting:Fewer features lead to simplermodels, reducing the risk of overfitting,
where a model learns noise in the training data instead of the underlying patterns.
3. Decreases Computational Cost:Working with a smallerset of features reduces the
computational resources required for training, leading to faster model training and evaluation.
4. Enhances Interpretability:A model with fewer featuresis often easier to understand and
interpret, making it more accessible for stakeholders.
Types of Feature Selection Methods
Feature selection methods can be broadly categorized into three types:
1. Filter Methods:
○ These methods evaluate the relevance of features using statistical measures without
involving any machine learning algorithms.
○ Examples include:
■ C orrelation Coefficient:Measures the linear relationship between features and
the target variable.
■ Chi-Squared Test:Assesses whether there is a significantassociation between
categorical features and the target variable.
2. Wrapper Methods:
○ These methods evaluate feature subsets by training a model and assessing its
performance based on a specific evaluation metric.
○ Examples include:
■ Recursive Feature Elimination (RFE):Iteratively removesthe least important
features based on model performance.
■ Forward Selection:Starts with an empty model andadds features one at a time
based on performance improvement.
. Embedded Methods:
3
○ These methods perform feature selection as part of the model training process. They
incorporate feature selection within the learning algorithm.
○ Examples include:
■ Lasso Regression:Adds a penalty to the loss functionthat encourages sparsity
in the model coefficients, effectively selecting features.
■ Decision Trees:Automatically perform feature selectionby splitting nodes
based on feature importance.
Example of Feature Selection
Consider a dataset used for predicting house prices with the following features:
● ize of the house (sq ft)
S
● Number of bedrooms
● Number of bathrooms
● Age of the house
● Proximity to schools
● Proximity to public transport
● Number of previous owners
Process of Feature Selection:
1. F ilter Methods:Use correlation coefficients to identifyfeatures most strongly correlated with
house prices, such as size, number of bedrooms, and proximity to schools.
2. Wrapper Methods:Apply recursive feature eliminationto find the best subset of features that
leads to the highest predictive accuracy.
3. Embedded Methods:Use Lasso regression to penalizeless important features, leading to a
model that only retains the most significant predictors.
e. Model Training and Evaluation
T
● rain various models using different sets of features.
● Evaluate model performance based on metrics appropriate for the problem (e.g., accuracy,
F1-score, RMSE).
f. Iteration
eature engineering is often an iterative process. Based on model performance, revisit previous steps,
F
refine features, and explore new ones.
6. Dimensionality Reduction Techniques
imensionality reduction techniques are essential in machine learning for simplifying models, reducing
D
computational cost, and improving visualization of high-dimensional data. This section covers four key
dimensionality reduction methods: Principal Component Analysis (PCA), Sparse PCA, Kernel PCA, and
T-distributed Stochastic Neighbor Embedding (T-SNE).
1. Principal Component Analysis (PCA)
verview:PCA is a linear dimensionality reductiontechnique that transforms the original features into
O
a new set of orthogonal features (called principal components) ordered by the amount of variance they
capture from the data.
How PCA Works:
1. S tandardization:Scale the data (if necessary) sothat each feature has a mean of zero and a
standard deviation of one. This step is crucial for PCA, as it is sensitive to the variance of the
features.
2. Covariance Matrix Computation:Calculate the covariancematrix of the standardized data to
understand how features vary together. The covariance matrix helps to identify correlations
between features.
3. Eigenvalue and Eigenvector Calculation:Compute theeigenvalues and eigenvectors of the
covariance matrix. Eigenvalues indicate the amount of variance captured by each principal
component, while eigenvectors define the direction of these components in the feature space.
4. Principal Components Selection:Sort the eigenvaluesin descending order and select the top
kkk eigenvectors that correspond to the largest eigenvalues. These eigenvectors form a new
feature space.
5. Transformation:Project the original data onto thenew feature space formed by the selected
principal components.
Advantages:
P
● CA is effective in reducing dimensionality while preserving as much variance as possible.
● It can help visualize high-dimensional data by projecting it into 2D or 3D.
Disadvantages:
● P CA assumes linear relationships among features, which may not capture complex patterns in
the data.
● It can be sensitive to outliers.
2. Sparse PCA
verview:Sparse PCA extends traditional PCA by incorporatinga sparsity constraint, encouraging the
O
selection of a small number of important features while capturing variance.
How Sparse PCA Works:
1. S parsity Constraint:Unlike standard PCA, which uses all features to compute the principal
components, Sparse PCA adds a constraint that limits the number of non-zero coefficients in
the eigenvectors. This leads to a simpler and more interpretable model.
2. Optimization Problem:The goal is to solve an optimizationproblem that minimizes the
reconstruction error while promoting sparsity. This is often achieved using techniques such as
Lasso regularization.
3. Feature Selection:By encouraging sparsity, SparsePCA naturally selects a few relevant
features that contribute significantly to the variance.
Advantages:
P
● roduces components that are easier to interpret due to fewer non-zero elements.
● Helps in situations where the number of features is much larger than the number of samples.
Disadvantages:
T
● he choice of sparsity parameter can be challenging and may require cross-validation.
● The algorithm can be computationally intensive compared to standard PCA.
3. Kernel PCA
verview:Kernel PCA is an extension of PCA that allowsfor non-linear dimensionality reduction by
O
applying the kernel trick. It enables the analysis of data in higher-dimensional feature spaces without
explicitly computing the coordinates of the data in that space.
How Kernel PCA Works:
1. K ernel Function:Choose a kernel function (e.g., polynomial,Gaussian RBF) that computes the
dot product of data points in a higher-dimensional space without explicitly transforming the
data. This function captures complex relationships among features.
2. Compute Kernel Matrix:Construct the kernel matrix,which contains the pairwise similarities
between all data points in the feature space defined by the kernel function.
3. Centering the Kernel Matrix:Center the kernel matrixby subtracting the mean from each row
and column to ensure that the kernelized data has a mean of zero.
4. Eigenvalue and Eigenvector Calculation:Calculatethe eigenvalues and eigenvectors of the
centered kernel matrix.
5. Projection:Project the original data into the newfeature space defined by the selected kernel
principal components.
Advantages:
K
● ernel PCA captures non-linear structures in data, making it suitable for complex datasets.
● It can reveal clusters and patterns that traditional PCA may miss.
Disadvantages:
● C omputationally expensive, especially for large datasets, as it requires constructing the kernel
matrix.
● The choice of the kernel function and its parameters can significantly affect results.
4. T-distributed Stochastic Neighbor Embedding (T-SNE)
verview:T-SNE is a non-linear dimensionality reductiontechnique primarily used for visualizing
O
high-dimensional datasets in two or three dimensions. It focuses on preserving local structures and
relationships between data points.
How Does t-SNE Work?
1. Start with High-Dimensional Data:
○ Imagine you have data with many features (dimensions), like a dataset of images with
thousands of pixels or a dataset of text with many words.
2. Calculate Pairwise Similarities:
○ t-SNE calculates the similarity between data points in the original high-dimensional
space. Points that are close to each other in the original space are considered similar.
3. Convert Similarities to Probabilities:
○ These similarities are converted into probabilities, which represent how likely it is that
two data points are neighbors.
4. Map to Lower-Dimensional Space:
○ t-SNE then tries to create a lower-dimensional map (like 2D or 3D) that preserves the
same neighborhood structure.
○ It uses a special cost function thatpenalizes incorrectdistances, ensuring that points
that were close in the original space stay close in the reduced space.
5. Optimization:
○ The algorithm goes through several iterations, gradually improving the layout in the
lower-dimensional space to better reflect the original high-dimensional relationships.
Key Points to Remember
t -SNE tries tokeep similar points close togetherand dissimilar points far apart.
●
● It’s mainly used forvisualization, not for predictive modeling.
● The output is usually a2D or 3D scatter plot, showing clusters or groups in your data.
Advantages:
● T -SNE is effective in preserving local structure, making it suitable for visualizing clusters and
patterns in complex datasets.
● It can reveal meaningful groupings and relationships that may not be apparent in higher
dimensions.
Disadvantages:
T
● -SNE is computationally intensive and may take a long time to process large datasets.
● It does not preserve global structures, meaning that distances in the low-dimensional space do
not represent distances in the high-dimensional space accurately.
● The results can be sensitive to the choice of hyperparameters (e.g., perplexity).
Are feature selection and dimensionality reduction related? elaborate
es,feature selectionanddimensionality reductionare related techniques used in data
Y
preprocessing. Both aim to reduce the complexity of a dataset by focusing on the most important
information. However, they achieve this goal in different ways.
○
Compare PCA and T-SNE in tabular form
Feature PCA (Principal Component t-SNE (t-distributed Stochastic
Analysis) Neighbor Embedding)
Type Linear dimensionality reduction on-linear dimensionality
N
reduction
Goal reserve global variance; reduce
P reserve local structure; visualize
P
dimensionality while retaining as high-dimensional data in lower
much information as possible dimensions
Output ew features (principal
N ew features that represent the
N
components) are linear original data in a
combinations of original features lower-dimensional space
Data Relationships C
aptures linear relationships aptures non-linear relationships
C
among features and local structures
omputational
C enerally faster and more efficient C
G omputationally intensive,
Complexity on large datasets due to linear especially for large datasets due to
nature pairwise distance calculations
Interpretability ew components can often be
N ew dimensions typically lack
N
interpreted in terms of original direct interpretability
features
ensitivity to
S ensitive to the scale of the
S enerally robust to the scale of the
G
Scaling features; standardization is usually features
required
Use Cases ata pre-processing, feature
D ata visualization, exploring
D
extraction, noise reduction complex datasets, clustering
Parameter Tuning inimal parameters; mostly
M ensitive to hyperparameters like
S
involves choosing the number of perplexity, which can greatly affect
components results
Visualization an project data into 2D or 3D but
C xcellent for visualizing clusters
E
may not reveal clusters effectively and relationships in
high-dimensional data
Explain with an example the importance of dimensionality reduction
imensionality reduction is a crucial technique in machine learning and data analysis, particularly
D
when working with high-dimensional datasets. It simplifies the dataset by reducing the number of
features while retaining as much information as possible. This process can improve model
performance, enhance visualization, and reduce computational costs. Let’s illustrate the importance of
dimensionality reduction using the example of a handwritten digits dataset.
Example: Handwritten Digits Dataset
Dataset Overview
he dataset consists of images of handwritten digits (0-9), with each image represented by pixel
T
values. Specifically, consider the following characteristics:
I mage Size:28x28 pixels
●
● Total Features:Each image is represented as a vectorof 784 features (since 28×28=784),
where each feature corresponds to the grayscale value of a pixel (0 for black, 255 for white).
Challenge
hen creating a machine learning model to classify these digits, several challenges arise due to the
W
high dimensionality:
● O verfitting:The model can easily memorize the trainingdata instead of learning general
patterns, which may result in poor performance on unseen data. With 784 features, the model
may capture noise rather than relevant features.
● Curse of Dimensionality:As the number of dimensionsincreases, the volume of the feature
space increases exponentially. This sparsity makes it difficult for the model to learn effectively,
as it requires more data to obtain reliable estimates.
● Computational Cost:High dimensionality increasesthe computational burden during training
and prediction, leading to longer processing times and more resource consumption.
Dimensionality Reduction Process
o address these challenges, we can apply dimensionality reduction techniques, such asPrincipal
T
Component Analysis (PCA), to transform the data:
1. Applying PCA:
○ PCA identifies the directions (principal components) in which the variance of the data is
maximized.
○ For the handwritten digits dataset, we can reduce the dimensionality from 784 to, for
example, 50 principal components, capturing the most significant variance in the
dataset.
2. Resulting Dimensions:
○ After applying PCA, we now have a reduced set of features (50 principal components)
that summarize the most important information in the original dataset.
Benefits After Dimensionality Reduction
1. Improved Model Accuracy:
○ With a reduced set of informative features, the model can generalize better and achieve
higher accuracy on unseen data, as it focuses on the most relevant patterns in the data
rather than noise.
2. Faster Training Time:
○ Reducing the dimensionality significantly decreases the training time of the model, as
fewer features mean less computational complexity. This makes the training process
more efficient.
3. Easier Visualization:
○ Dimensionality reduction allows for better visualization of high-dimensional data. For
example, by further reducing the dimensions to 2 or 3 using PCA, we can plot the digits
on a 2D or 3D graph.
○ In a 2D scatter plot, each point represents an image of a handwritten digit, color-coded
by digit class. This visualization helps identify clusters of similar digits and understand
the data structure.
4. Enhanced Interpretability:
○ With fewer dimensions, the model becomes easier to interpret. We can analyze the
principal components to understand which pixel combinations are significant for
distinguishing between different digits.
Visual Representation
● B efore Dimensionality Reduction:Visualizing the dataset in 784-dimensional space is
impossible. The high dimensionality complicates understanding the data distribution and
relationships among different classes.
● After Dimensionality Reduction:After applying PCA,we can create a 2D scatter plot of the 50
principal components, allowing us to visually identify clusters of similar digits, facilitating a
deeper understanding of how the digits relate to one another.
Conclusion
In summary, dimensionality reduction plays a vital role in improving the efficiency and effectiveness of
machine learning models, especially in high-dimensional datasets like handwritten digits. By
simplifying the data while retaining essential information, we enhance model accuracy, reduce
computational costs, and gain valuable insights through visualization.
8. Multiple Regression
Definition
ultiple Linear Regression (MLR) is a statistical method used to model the relationship between a
M
single dependent variable and two or more independent variables. It allows us to understand how
multiple factors simultaneously influence an outcome, providing a nuanced view that simple linear
regression cannot achieve.
Mathematical Representation
Key Assumptions
For MLR results to be valid, several assumptions must be satisfied:
1. L inearity: The relationship between independent variablesand the dependent variable is linear.
This can be checked using scatter plots or residual plots.
2. Independence: The residuals (errors) should be independentof each other. This is particularly
important for time series data where values can be correlated.
3. H omoscedasticity: The residuals should have constant variance at all levels of the independent
variables. Violation of this assumption can lead to inefficient estimates.
4. Normality: The residuals should be approximately normallydistributed, especially for
hypothesis testing.
5. No Multicollinearity: The independent variables shouldnot be highly correlated with each other,
as this can inflate the variance of coefficient estimates and make them unstable.
Steps to Perform Multiple Linear Regression
. D
1 ata Collection: Gather relevant data for the dependentand independent variables.
2. Data Preprocessing:
○ Handle missing values (e.g., imputation or removal).
○ Encode categorical variables (e.g., one-hot encoding).
○ Scale or standardize numerical features if necessary.
3. Exploratory Data Analysis (EDA): Use visualizations (scatter plots, correlation matrices) and
summary statistics to understand relationships and distributions within the data.
4. Model Fitting: Use statistical software (e.g., Pythonstatsmodels or scikit-learn, R) to fit the
MLR model to the dataset.
5. Model Evaluation:
○ Assess Model Fit: Use metrics like R-squared and AdjustedR-squared to evaluate how
well the model explains the variance in the dependent variable.
○ Mean Squared Error (MSE): Calculate the average squareddifferences between
observed and predicted values to measure accuracy.
○ Residual Analysis: Check for violations of the assumptions(e.g., normality,
homoscedasticity) using residual plots.
6. Interpretation of Results: Analyze the coefficients to understand the significance and impact of
each independent variable on the dependent variable.
7. Prediction: Use the fitted model to make predictionson new or unseen data.
Example of Multiple Linear Regression
Scenario: Predicting Employee Salary
uppose you want to predict the annual salary of employees in a tech company based on several
S
features. The dataset includes:
D
● ependent Variable (Y): Employee Salary (in dollars).
● Independent Variables:
○ Years of Experience (X1): The number of years theemployee has worked.
○ Education Level (X2): Encoded as a categorical variable(1 for Bachelor’s, 2 for Master’s,
3 for PhD).
○ Age of the Employee (X3): Measured in years.
○ Number of Certifications (X4): The number of relevantcertifications the employee
holds.
Interpretation of Coefficients:
● Y ears of Experience: For every additional year ofexperience, the employee’s salary increases
by $5,000, holding other factors constant.
● Education Level: Each step up in education level (e.g.,from Bachelor’s to Master’s) adds
$10,000 to the salary.
● Age: Each additional year of age corresponds to a $2,000 increase in salary.
● Number of Certifications: Each additional certificationadds $3,000 to the salary.
Model Evaluation:
● R -squared: Indicates the proportion of variance inemployee salaries explained by the
independent variables (e.g., an R-squared value of 0.78 means 78% of the variability in salaries
can be explained by the model).
● Mean Squared Error (MSE): A lower MSE indicates betterpredictive accuracy of the model.
● Residual Analysis: Plot residuals to check for randomnessand homoscedasticity.
Conclusion
ultiple Linear Regression is a powerful technique for modeling the relationship between multiple
M
predictors and a single outcome. In this example, we explored how years of experience, education
level, age, and certifications affect employee salaries in a tech company. By validating assumptions
and interpreting results carefully, stakeholders can derive meaningful insights and make informed
decisions based on the regression model.
9. Logistic Regression
efinition:
D
Logistic Regression is a statistical method for binary classification problems, where the outcome
variable is categorical with two possible outcomes (e.g., yes/no, success/failure). Unlike linear
regression, which predicts continuous outcomes, logistic regression estimates the probability that a
given input point belongs to a particular category.
Mathematical Representation
he relationship between the independent variables and the log-odds of the dependent variable is
T
modeled using the logistic function. The probability of the outcome being 1 (e.g., success) is
expressed as:
Key Characteristics of Logistic Regression
1. B inary Outcomes: Primarily used for situations where the dependent variable has two possible
outcomes.
2. Odds and Log-Odds: The model predicts the odds ofthe dependent variable being 1 versus 0.
The log-odds transformation is employed to linearize the relationship.
3. Non-linear Decision Boundary: The logistic functionallows for a non-linear relationship
between independent variables and the probability of the dependent variable.
Assumptions of Logistic Regression
For the results of Logistic Regression to be valid, certain assumptions must be met:
. B
1 inary Outcome: The dependent variable must be binary(0 or 1).
2. Independence of Errors: Observations should be independentof each other.
3. Linearity of Logits: The log-odds of the dependent variable should have a linear relationship
with the independent variables.
4. No Multicollinearity: Independent variables shouldnot be too highly correlated with each other,
as this can inflate the variance of the coefficient estimates.
Steps to Perform Logistic Regression
. D
1 ata Collection: Gather data for the dependent binaryvariable and independent variables.
2. Data Preprocessing: Handle missing values, encodecategorical variables, and normalize
numerical features if necessary.
3. Exploratory Data Analysis (EDA): Understand the datathrough visualizations and summary
statistics to identify patterns or anomalies.
4. Model Fitting: Use statistical software or programminglanguages (like Python or R) to fit the
logistic regression model to the data.
5. Model Evaluation: Assess the model’s performance usingmetrics like accuracy, precision,
recall, F1-score, and the ROC curve.
6. I nterpretation of Results: Analyze the coefficients to understand the impact of each
independent variable on the likelihood of the outcome.
7. Prediction: Use the model to predict the probabilitiesof the binary outcome for new data.
Example: Predicting Heart Disease
cenario: Suppose we want to predict whether a patienthas heart disease (1) or not (0) based on
S
several health-related features.
D
● ependent Variable: Heart Disease Status (Y).
● Independent Variables:
○ Age (X1)
○ Cholesterol Level (X2)
○ Blood Pressure (X3)
○ Body Mass Index (BMI) (X4)
Model Equation:
● A ge: For every additional year in age, the log-oddsof having heart disease increase by 0.04,
holding other factors constant.
● Cholesterol: For every additional unit increase incholesterol, the log-odds of having heart
disease increase by 0.02, holding other factors constant.
● Blood Pressure: For every unit increase in blood pressure, the log-odds of having heart disease
increase by 0.03.
● BMI: For every additional unit increase in BMI, thelog-odds of having heart disease increase by
0.01.
Model Evaluation
To evaluate the model's performance, we can use several metrics:
● C onfusion Matrix: To assess the true positives (TP),true negatives (TN), false positives (FP),
and false negatives (FN).
● Accuracy: The proportion of correct predictions outof the total predictions.
● ROC Curve: To analyze the trade-off between sensitivity(true positive rate) and specificity (true
negative rate).
y interpreting the results and evaluating the model, healthcare professionals can make informed
B
decisions based on the likelihood of a patient developing heart disease, thereby improving preventive
care and treatment strategies.
10. Residual Analysis
efinition:
D
Residual analysis is the examination of residuals (the differences between observed and predicted
values) in regression models. It serves as a diagnostic tool to evaluate the fit of a model and the
validity of its underlying assumptions. By analyzing residuals, you can identify potential issues with the
model, such as non-linearity, heteroscedasticity, and the presence of outliers.
Key Concepts in Residual Analysis
1. Residuals:
○ Definition:A residual is the difference between the actual value (yiy_iyi) and the
predicted value (y^i\hat{y}_iy^i) for an observation iii. It is calculated using the formula:
2. Purpose of Residual Analysis:
○ Model Fit Assessment:Residuals help determine howwell the regression model fits the
data. Ideally, residuals should be randomly distributed around zero.
○ Validation of Assumptions:Residual analysis is usedto validate the key assumptions
of regression analysis, including linearity, independence, homoscedasticity, and
normality.
○ Detection of Outliers:Residuals can indicate outliersor influential points that may
disproportionately affect the regression model.
Key Assumptions of Regression Analysis
Residual analysis helps verify several key assumptions:
1. Linearity:
○ The relationship between the independent variables and the dependent variable should
be linear.
○ Check:By plotting residuals against predicted values,you can assess whether the
residuals exhibit a random scatter. A discernible pattern (e.g., a curve) suggests
non-linearity.
2. Independence:
○ Residuals should be independent of each other, particularly in time series data.
○ Check:For time series data, residuals can be plottedagainst time to identify patterns
that indicate autocorrelation.
3. Homoscedasticity:
○ T he residuals should have constant variance across all levels of the independent
variables.
○ Check:Plotting residuals against predicted valuesor any independent variable helps
visualize the spread. If the spread appears to increase or decrease with the fitted
values, it indicates heteroscedasticity (non-constant variance).
. Normality:
4
○ Residuals should be approximately normally distributed, particularly for small sample
sizes.
○ Check:Histograms or Q-Q plots of residuals can beused to assess normality. Ideally,
the residuals should form a bell-shaped distribution in a histogram and closely follow a
straight line in a Q-Q plot.
Steps for Conducting Residual Analysis
1. C alculate Residuals:
Compute residuals for each observation using the residual formula. This can be done after
fitting the regression model.
2. Create Residual Plots:
○ Residual vs. Fitted Plot:Plot the residuals againstthe predicted values. This plot helps
assess both linearity and homoscedasticity.
○ Normal Q-Q Plot:This plot is used to check if residualsfollow a normal distribution.
Points should closely align along the diagonal line if normality holds.
○ Histogram of Residuals:Create a histogram to visualizethe distribution of residuals,
looking for symmetry around zero.
3. Conduct Statistical Tests:
Statistical tests can provide more formal assessments:
○ Normality Tests:Use tests such as the Shapiro-Wilktest to check for normality of the
residuals.
○ Independence Tests:The Durbin-Watson test can beused to check for independence of
residuals, particularly in time series data.
4. Identify Outliers:
Investigate any residuals that are significantly larger or smaller than the others. These outliers
may indicate influential data points that could affect the model.
5. Model Refinement:
Based on the findings from residual analysis, consider the following:
○ If assumptions are violated (e.g., evidence of nonlinearity or heteroscedasticity), explore
potential remedies, such as transforming variables, adding interaction terms, or
selecting a different modeling approach (e.g., polynomial regression or generalized
additive models).
Conclusion
esidual analysis is a critical component of regression analysis. It allows you to evaluate the fit of your
R
model and validate the assumptions underlying the regression analysis. By examining the residuals,
you can identify potential issues that may affect the model's accuracy and reliability, enabling you to
refine the model for better predictive performance. Properly conducting residual analysis leads to
more trustworthy insights and conclusions drawn from your regression analysis.
nit 2
U
Overfitting
verfittingis a common issue in machine learning and statistical modeling where a model learns the
O
noise and details of the training data to the extent that it negatively impacts its performance on new,
unseen data. An overfitted model is excessively complex, capturing patterns that do not generalize
well beyond the training dataset.
Understanding Overfitting
To better understand overfitting, consider the following:
1. Training vs. Testing:
○ Training Data: The subset of data used to train themodel.
○ Testing Data: A separate subset used to evaluate themodel’s performance.
○ An effective model should perform well on both datasets, indicating it has learned the
underlying patterns rather than the specifics of the training data.
2. Model Complexity:
○ Simple models (e.g., linear regression with a few features) may underfit, failing to
capture the underlying trends.
○ Complex models (e.g., deep neural networks or decision trees with many branches) risk
overfitting if they become too tailored to the training data.
Signs of Overfitting
1. H igh Training Accuracy, Low Testing Accuracy: Themodel shows excellent performance on
the training dataset but significantly worse performance on the testing dataset.
2. Complexity Indicators: The model's parameters or structurebecome excessively complex,
such as having too many features or overly intricate decision boundaries.
3. Variance: A model that overfits tends to have highvariance, meaning small changes in the
training data can lead to substantial changes in the model's predictions.
Causes of Overfitting
1. M odel Complexity: Using overly complex models withtoo many parameters relative to the
amount of training data can lead to overfitting.
2. Insufficient Training Data: A small training datasetmay not capture the underlying distribution,
leading the model to learn noise rather than true patterns.
3. Noise in Data: High levels of noise or outliers inthe training data can mislead the model into
capturing irrelevant patterns.
4. Lack of Regularization: When regularization techniquesare not employed, the model may fit
the training data too closely.
Consequences of Overfitting
1. P oor Generalization: The model performs well on the training set but poorly on unseen data,
making it unreliable for real-world applications.
2. Increased Error: Overfitting can lead to increasedprediction error, particularly on new data.
3. Misleading Insights: Decisions based on overfittedmodels can lead to incorrect conclusions or
actions.
Preventing Overfitting
Several techniques can be employed to mitigate overfitting:
1. Simplifying the Model:
○ U se simpler models with fewer parameters (e.g., linear models instead of polynomial
models).
○ Prune complex models (e.g., decision trees) to remove less significant branches.
2. Regularization:
○ L1 Regularization (Lasso): Adds a penalty equal tothe absolute value of the
coefficients to the loss function, encouraging sparsity.
○ L2 Regularization (Ridge): Adds a penalty equal tothe square of the coefficients to the
loss function, discouraging large coefficients.
3. Cross-Validation:
○ Use techniques like k-fold cross-validation to ensure the model's performance is
consistent across different subsets of data.
○ Helps in assessing how well the model generalizes to unseen data.
4. Increasing Training Data:
○ Collect more training data to help the model learn the underlying distribution better.
○ Data augmentation techniques can also be used to artificially increase the size of the
dataset.
5. Early Stopping:
○ Monitor model performance on a validation set during training and stop training when
performance begins to degrade.
○ Useful in iterative training methods like gradient descent.
6. Dropout (for Neural Networks):
○ Randomly drop units (neurons) during training to prevent reliance on specific neurons,
helping to generalize better.
Visual Representation of Overfitting
common way to visualize overfitting is through a learning curve, which plots training and validation
A
errors over time or model complexity. In an overfitting scenario, the training error decreases while the
validation error increases after a certain point.
Explain truncating and pruning in decision trees
1. Truncating
runcatinga decision tree involves limiting its depthor the number of splits during the tree-building
T
process. This approach effectively reduces the size of the tree from the outset.
Key Characteristics:
● D epth Limitation: The maximum depth of the tree isdefined before training, ensuring that the
tree does not grow too deep. For example, setting a maximum depth of 3 means that the tree
can have at most three splits from the root to the leaves.
● Feature Limitation: Instead of allowing the model to use all features, truncating can also
involve restricting the number of features considered for splitting at each node.
● Control Overfitting: By truncating, we can controloverfitting right from the beginning, as the
tree is not allowed to explore all possible splits.
Advantages:
R
● educes the risk of overfitting by limiting complexity.
● Increases computational efficiency, as a smaller tree is easier and faster to evaluate.
Disadvantages:
● May lead to underfitting if the tree is too shallow, missing important patterns in the data.
2. Pruning
runingis the process of removing branches from afully grown decision tree after it has been built.
P
The goal is to reduce the size of the tree by eliminating nodes that provide little predictive power, thus
simplifying the model.
Types of Pruning:
1. Pre-Pruning(Early Stopping):
○ This involves stopping the growth of the tree before it reaches its full depth.
○ A predefined condition (like a minimum number of samples required to split a node or a
maximum depth) is used to decide whether to split further or not.
○ Example: If a node has fewer than 10 samples, it may not be split any further.
2. Post-Pruning:
○ In this method, the tree is fully grown first, and then branches are removed based on
specific criteria.
○ Pruning is typically done using validation data to assess whether the removal of a
branch improves generalization.
○ Common techniques include:
■ Cost Complexity Pruning: Balances tree size and predictionerror by evaluating
the trade-off between the size of the tree and its accuracy on the training data.
■ Minimum Description Length (MDL): Chooses a subtreethat minimizes the total
description length (model complexity plus error).
Advantages:
E
● nhances model interpretability by reducing complexity.
● Improves the model's performance on unseen data by reducing overfitting.
Disadvantages:
R
● equires additional computation for pruning decisions, especially in post-pruning.
● If too much pruning occurs, it can lead to underfitting.
Comparison of Truncating and Pruning
Aspect Truncating Pruning
Timing Done during tree construction Done after the tree is fully built
Method imits depth or features
L emoves branches based on
R
upfront evaluation
Impact on Complexity R
educes complexity from the Simplifies a fully grown tree
start
Risk of Overfitting ess risk due to early
L ddresses overfitting after it
A
limitations occurs
Control Less control over tree structure More control by evaluating splits
Comparison: SVM vs. Decision Tree Classifiers
Feature Support Vector Machine (SVM) Decision Tree
Approach inds a hyperplane that maximizes
F ecursively splits the data into
R
the margin between classes. subsets based on feature
values to create a tree structure.
Interpretability omplex to interpret, especially
C asy to interpret and visualize;
E
with non-linear kernels. decisions are rule-based.
andling of
H ses kernel tricks (e.g., RBF kernel) H
U andles non-linearity naturally
Non-linear Data to map data to higher dimensions. by splitting on feature
thresholds.
Overfitting ess prone to overfitting due to the
L rone to overfitting if the tree is
P
margin-maximizing approach deep; can be mitigated with
(controlled by regularization pruning or setting maximum
parameter CCC). depth.
Training Time enerally slower, especially for
G ast to train; efficient for large
F
large datasets or with complex datasets.
kernels.
erformance on
P ensitive to outliers since the
S ore robust to outliers but can
M
Noisy Data decision boundary is determined by still overfit noisy data.
support vectors.
andling of
H orks well in high-dimensional
W ay struggle with very
M
High-dimensional space due to the kernel trick. high-dimensional data, though
Data feature importance can help.
Scalability oor scalability for very large
P ighly scalable; works well for
H
datasets. both small and large datasets.
Multiclass Support equires strategies like One-vs-One N
R aturally handles multiclass
or One-vs-Rest for multiclass classification.
classification.
Maximum Margin Classifier in Support Vector Machines (SVM)
upport Vector Machines (SVM)are supervised learningalgorithms used for classification and
S
regression tasks. One of the key concepts in SVM is themaximum margin classifier, which focuses on
finding the best hyperplane that separates different classes in the feature space. This approach is
particularly effective in high-dimensional spaces and helps achieve good generalization performance.
Key Concepts
1. Hyperplane:
○ In an n-dimensional space, a hyperplane is a flat affine subspace of dimension n-1. For
example, in a two-dimensional space, a hyperplane is a line, while in three dimensions, it
is a plane.
○ The equation of a hyperplane can be represented as: w⋅x+b=0w \cdot x + b = 0w⋅x+b=0
where www is the weight vector, xxx is the feature vector, and bbb is the bias.
2. Classes:
○ In a binary classification scenario, the goal is to separate two classes (labeled as +1
and -1) using a hyperplane.
○ The points that are closest to the hyperplane and influence its position are called
support vectors.
. Margin:
3
○ Themarginis defined as the distance between the hyperplane and the closest points
from either class. The aim is to maximize this margin.
○ A larger margin indicates better separation between the classes, which leads to better
generalization on unseen data.
Finding the Maximum Margin Hyperplane
he goal of the maximum margin classifier is to find the hyperplane that maximizes the margin while
T
correctly classifying the training data. This can be formulated as an optimization problem:
Geometric Interpretation
1. Support Vectors:
○ The support vectors are the data points that lie closest to the hyperplane. These points
directly influence the position and orientation of the hyperplane.
○ If the support vectors are removed or changed, the position of the hyperplane may also
change, while points further away from the hyperplane do not affect it.
2. Visualization:
○ In a two-dimensional feature space, the maximum margin hyperplane can be visualized
as a line that separates two classes with the maximum distance (margin) between the
closest points from each class.
Advantages of Maximum Margin Classifier
1. R obustness: The maximum margin classifier is robustto overfitting, especially in
high-dimensional spaces, as it focuses on the most critical points (support vectors) for
defining the decision boundary.
2. Generalization: By maximizing the margin, the modelaims to generalize better on unseen data,
leading to improved classification performance.
Limitations
1. L inearly Separable Data: The maximum margin classifier works best with linearly separable
data. In cases where classes are not linearly separable, SVMs can still be employed by using
kernel functions.
2. Computational Complexity: The optimization problemcan become computationally intensive,
especially with large datasets.
SVM Kernels
ernels are functions that transform the input data into a higher-dimensional space, allowing the SVM
K
to find a linear separation in a space where the data may not originally be linearly separable. Different
kernel functions are used to handle different types of data. The most commonly used kernels are:
1. Linear Kernel:
○ The linear kernel is used when the data is linearly separable. It simply computes the dot
product between the input features.
○
2. Polynomial Kernel:
○ The polynomial kernel allows SVM to fit the hyperplane in a higher-dimensional space,
making it suitable for more complex datasets.
○
3. Radial Basis Function (RBF) Kernel:
○ The RBF (or Gaussian) kernel is widely used for non-linear classification. It maps the
data into an infinite-dimensional space.
○
. Sigmoid Kernel:
4
○ The sigmoid kernel behaves similarly to neural networks and can be used for non-linear
problems.
○
What is Ensemble Learning?
nsemble Learningis a machine learning paradigm wheremultiple models (often called "weak
E
learners") are trained to solve the same problem, and their predictions are combined to improve the
overall performance. The idea behind ensemble learning is that a group of weak models can come
together to form a strong model, reducing variance, bias, or improving predictions.
he main goal is to make better predictions by averaging or combining multiple models to produce
T
more accurate and stable results.
Ensemble learning types:-
Feature Bagging Boosting Stacking
Definition C
ombines predictions equentially builds
S ombines multiple
C
from multiple models models, where each new models (base learners)
trained independently on model attempts to correct using a meta-model to
random subsets of data. errors made by previous improve predictions.
models.
ype of
T arallel ensemble
P equential ensemble
S ybrid ensemble
H
Ensemble (independent training) (dependent training) (layered approach)
odel
M odels are trained
M odels are built
M ase models can be
B
Independence independently; each sequentially; each new trained independently; a
model is built using a model is trained based meta-model learns how
random sample of the on the performance of to best combine their
training data. prior models. predictions.
andling
H educes variance by
R educes bias by
R tilizes predictions from
U
of Errors averaging predictions; focusing on the errors of multiple base models and
does not focus on previous models, can leverage their
errors made by improving performance strengths; can reduce
individual models. iteratively. both bias and variance.
Overfitting ess prone to
L ore prone to overfitting if
M an help mitigate
C
overfitting due to not regularized properly overfitting by leveraging
averaging since models are built diverse models and
predictions. sequentially. combining their outputs.
Complexity elatively simple to
R ore complex due to
M ore complex as it
M
implement; requires the sequential training involves multiple levels of
fewer hyperparameters. and tuning of multiple models and a meta-model
models. for final predictions.
Examples R
andom Forest, daBoost, Gradient
A tacked Generalization
S
Bagged Decision Boosting Machines (GBM), (Stacking), Super Learner
Trees XGBoost
se
U ffective for reducing E
E ffective for improving seful when combining diverse
U
Cases variance in accuracy and reducing models to improve overall
high-variance bias in weak learners. predictive performance.
models.
rediction
P verage or majority W
A eighted sum of model eta-model combines the
M
Method vote of individual predictions, with more predictions from base
model predictions. weight given to models models, often using
that perform better. regression or another
learning algorithm.
Random Forest
Random Forestis a machine learning algorithm thatcreates a "forest" of multiple decision trees.
It combines the predictions of several decision trees to make a final prediction.
It can be used forclassification(predicting categories)andregression(predicting values).
he key idea is toreduce overfitting(when a modelis too specific to the training data) by using many
T
trees.
Key
Concepts in Random Forest:
● D ecision Trees: Each tree in the forest is a decisiontree trained on a random subset of the
data.
● Random Subsets: Random Forest selects random samples(bootstrapped datasets) from the
training data to build each decision tree.
● Random Features: It also randomly selects a subsetof features for splitting at each node in the
decision trees.
● Prediction Aggregation: For classification tasks,Random Forest uses majority voting among
trees for the final prediction, and for regression tasks, it averages the predictions of each tree.
Advantages of Random Forest:
● It reduces overfitting by averaging multiple decision trees.
● It is less sensitive to noisy data.
● It can handle large datasets with higher dimensionality.
● It provides feature importance scores, making it useful for feature selection.
Example of Random Forest
et’s consider a hypothetical scenario where a company wants to predict whether a customer will buy
L
a product based on several features.
Scenario:
The dataset includes information on customers with the following features:
● Age: The customer's age.
I ncome: The customer's income.
●
● Gender: The customer's gender (Male/Female).
● Purchased: The target variable indicating whetherthe customer purchased the product (1 for
Yes, 0 for No).
Steps Involved:
1. Data Preparation:
○ The company collects data from previous customers and organizes it into a structured
dataset. Categorical variables, such as Gender, might need to be encoded into
numerical formats (e.g., Male = 0, Female = 1) for the model to process.
2. Data Splitting:
○ The dataset is split into two parts: one for training the model (e.g., 80% of the data) and
one for testing its performance (e.g., 20% of the data).
3. Model Training:
○ A Random Forest model is created, which will consist of multiple decision trees. Each
tree is trained on different subsets of the data, allowing the model to learn various
patterns from the dataset.
4. Making Predictions:
○ After training, the model can make predictions on the test data. For instance, given a
new customer's age, income, and gender, the model will predict whether they are likely
to purchase the product.
5. Model Evaluation:
○ The predictions made by the Random Forest model are compared against the actual
outcomes in the test dataset. Key metrics such as accuracy (the proportion of correct
predictions), precision, recall, and F1-score are calculated to evaluate the model’s
performance.
Give the relevance of ROC curves and show its formula derivation.
Relevance of ROC Curves
eceiver Operating Characteristic (ROC) curvesarea fundamental tool used in evaluating the
R
performance of binary classifiers. They provide a visual representation of a model's diagnostic ability
across various threshold settings. Here are key aspects of their relevance:
1. Performance Measurement:
○ ROC curves help assess the trade-offs between true positive rates (sensitivity) and false
positive rates (1 - specificity) at different threshold levels.
2. Threshold Selection:
○ By analyzing the ROC curve, practitioners can select the optimal threshold for
classifying positive and negative instances based on the desired balance between
sensitivity and specificity.
3. Comparison of Classifiers:
○ ROC curves enable the comparison of multiple classifiers visually. The classifier with
the curve closer to the top-left corner generally has better performance.
4. Area Under the Curve (AUC):
○ TheArea Under the ROC Curve (AUC)quantifies theoverall ability of the model to
discriminate between positive and negative classes. An AUC of 1 indicates perfect
classification, while an AUC of 0.5 indicates no discriminative ability (random
guessing).
5. Independence from Class Distribution:
○ R
OC curves are invariant to changes in the class distribution, making them particularly
useful in imbalanced classification problems.
ROC Curve Definition and Components
1. T
rue Positive Rate (TPR): Also known as sensitivityor recall, TPR is the proportion of actual
positives that are correctly identified.
Interpretation: A higher TPR indicates better performancein identifying positive cases.
2. F
alse Positive Rate (FPR): The proportion of actualnegatives that are incorrectly classified as
positives.
I nterpretation: A lower FPR indicates fewer negativeinstances being incorrectly classified as
positive, which is desirable for a good classifier.
Plotting the ROC Curve
● Axes:
○ Thex-axisrepresents theFalse Positive Rate (FPR).
○ They-axisrepresents theTrue Positive Rate (TPR).
● Curve: The curve is generated by plotting the TPRagainst the FPR at different classification
thresholds. Each point on the curve corresponds to a specific threshold, illustrating the model's
performance across various levels of sensitivity and specificity.
Area Under the Curve
● T
heArea Under the Curve (AUC)quantifies the overallperformance of the classifier. AUC
values range from 0 to 1:
○ AUC = 1: Perfect classification.
○ AUC = 0.5: Model performs no better than random guessing.
○ AUC < 0.5: The model is performing worse than randomguessing.
Derivation of ROC Curve Formula
o derive the equations used to plot the ROC (Receiver Operating Characteristic) curve, we need to
T
understand the relationship between the True Positive Rate (TPR) and the False Positive Rate (FPR)
based on various classification thresholds.
Summary
he ROC curve is a graphical representation that helps to evaluate the performance of a binary
T
classifier across different thresholds. It provides insights into the classifier's ability to distinguish
between positive and negative classes. By analyzing the TPR and FPR at multiple thresholds, one can
select an optimal threshold that balances the trade-off between sensitivity and specificity. The area
under the ROC curve (AUC) is also commonly used to quantify the overall performance of the classifier,
with a higher AUC indicating better performance.
. Imagine a scenario of disease detection application where the dataset includes a historical
Q
dataset of 50 different people. Discuss which ensemble approach is better applicable if an
imbalanced dataset is observed.
Ans:
hoosingBoostingfor disease detection, particularlyin the context of an imbalanced dataset, is a
C
strategic decision backed by its unique advantages. Here’s a detailed explanation of why Boosting is
preferable, along with an example to illustrate its effectiveness.’
Detailed Example Scenario
Context
et's consider a hypothetical scenario involving a dataset with health records from50 patients, of
L
which45 are healthy(majority class) and5 have aspecific disease(minority class). This 90:10 ratio
presents a significant challenge for disease detection.
Steps in Applying Boosting
1. Choosing the Algorithm
○ SelectXGBoost, known for its efficiency and flexibility,especially in handling
imbalanced datasets.
2. Preprocessing the Data
○ Data Cleaning: Remove duplicates, handle missing values,and ensure all records are
relevant.
○ Feature Selection: Identify key features such as age,symptoms, blood tests, and
medical history that might influence the diagnosis.
○ Imbalance Handling:
■ Class Weighting: Assign a higher weight to the diseaseclass in the loss
function.
■ Resampling: Optionally apply SMOTE to generate syntheticexamples of the
minority class.
3. Splitting the Dataset
○ Divide the dataset into a training set (e.g., 80% of 50 patients = 40 patients) and a
testing set (20% = 10 patients) to assess model performance.
4. Training the Boosting Model
○ Fit theXGBoostmodel to the training data. The modelwill create an ensemble of trees:
■ First Iteration: A simple tree classifies the trainingset. Let’s say it correctly
identifies 4 of the 5 disease cases, but misclassified some healthy patients.
■ Weight Adjustment: The misclassified disease caseand any misclassified
healthy patients are given more weight in the next iteration.
■ Subsequent Iterations: Each new tree is trained tocorrect the mistakes of
previous trees, progressively improving the classification of the disease cases.
5. Model Evaluation
○ After training, evaluate the model using the testing set. Calculate key metrics:
■ Accuracy: The overall proportion of correct predictions.
■ Precision: The proportion of true positive diseasecases among all predicted
positive cases.
■ Recall (Sensitivity): The proportion of true positivedisease cases among all
actual disease cases.
■ F1-Score: The harmonic mean of precision and recall.
Example Results
Assume the evaluation yields the following results:
● ccuracy: 88%
A
● Precision: 90% (out of 10 predicted positive cases,9 were true positives)
● Recall: 80% (out of 5 actual disease cases, 4 werecorrectly predicted)
● F1-Score: 84% (balancing precision and recall)
6. Hyperparameter Tuning
○ Use techniques such as Grid Search or Random Search to optimize hyperparameters
like:
■ Learning Rate: Controls the contribution of each tree.
■ Number of Estimators: The number of trees in the ensemble.
■ Max Depth: Depth of each tree to avoid overfitting.
7. Testing the Model
○ Apply the trained model to new patient records to predict disease presence. Ensure to
evaluate predictions using the confusion matrix to analyze true positives, false
positives, true negatives, and false negatives.
. Interpreting Results
8
○ Utilize SHAP values or feature importance plots to understand which features most
significantly influenced the model's predictions. This can help identify critical risk
factors and assist in clinical decision-making.
Conclusion
oosting is a powerful technique for disease detection, particularly in imbalanced datasets. Its ability
B
to focus on misclassified instances and its iterative improvement mechanism leads to better detection
of minority classes, such as patients with diseases. The detailed steps illustrate how Boosting can be
effectively applied to achieve high accuracy, precision, and recall, making it a preferred choice for
medical applications where identifying the presence of a disease is critical for patient outcomes.
Q.Given a scenario of rainfall dataset . ExplainSVM algorithm implementation on it
Ans:
Simplified Dataset:
Day Temperature Humidity (%) Wind Speed Pressure Rainfall
(°C) (km/h) (hPa) (Yes/No)
1 28 65 12 1015 No
2 30 75 10 1005 Yes
3 29 60 15 1020 No
4 25 80 8 1000 Yes
5 32 70 13 1010 No
Step-by-Step Explanation of SVM Implementation:
1. Understanding the Problem
We are trying to predictwhether it will rain (Yes/No)based on weather features:
● emperature(°C)
T
● Humidity(%)
● Wind Speed(km/h)
● Atmospheric Pressure(hPa)
● Rainfall(Yes/No) – the target variable.
2. Preprocessing the Data
● L abel Encoding: Convert the target variable ("Rainfall")from categorical values ("Yes"/"No") to
numerical values. For example:
○ "Yes" = 1
○ "No" = 0
● After encoding, the dataset looks like this:
Day Temperature (°C) Humidity (%) Wind Speed Pressure Rainfall
(km/h) (hPa) (0/1)
1 28 65 12 1015 0
2 30 75 10 1005 1
3 29 60 15 1020 0
4 25 80 8 1000 1
5 32 70 13 1010 0
● F
eature Scaling: SVM is sensitive to the scale offeatures, so we apply scaling to standardize
the values of all features. This ensures that temperature, humidity, wind speed, and pressure
are on the same scale.
3. Splitting the Data
Before training the model, we need to divide the data into two sets:
T
● raining Set: Used to train the SVM model (e.g., 80%of the data).
● Test Set: Used to evaluate the model's performance(e.g., 20% of the data).
Since our dataset is small, we can use cross-validation to maximize the training data available.
4. Choosing the Kernel Function
VM requires selecting akernel function. In thiscase, since we suspect that weather features like
S
temperature, humidity, and pressure have complex (possibly nonlinear) relationships, we choose the
Radial Basis Function (RBF) kernel.
he RBF kernel maps the data into a higher-dimensional space, making it easier to find a decision
T
boundary (hyperplane) that separates rainy and non-rainy days.
5. Training the SVM Model
The SVM model is trained using the training data. During training, the algorithm:
Identifies thesupport vectors(the data points closestto the decision boundary).
●
● Attempts to find theoptimal hyperplanethat bestseparates the two classes (rain vs. no rain)
while maximizing themargin(the distance betweenthe hyperplane and the nearest points of
both classes).
The result is a decision boundary that separates the two classes based on the weather data.
6. Making Predictions
fter the model is trained, you can use it to make predictions on new weather data. For example, let’s
A
take a new weather observation:
● emperature: 29°C
T
● Humidity: 72%
● Wind Speed: 11 km/h
● Pressure: 1012 hPa
he trained SVM model would use these inputs to predict whether it will rain (Yes/No) based on the
T
decision boundary it learned during training.
7. Evaluating the Model
o evaluate how well the SVM model performs, you can use the test data (which was not used in
T
training). You can measure:
A
● ccuracy: The percentage of correct predictions.
● Confusion Matrix: Shows the number of true positives(correctly predicted rainy days), true
negatives (correctly predicted non-rainy days), false positives, and false negatives.
● Precision and Recall: These metrics help understandhow well the model detects rainy days
and whether it misses any rainy days.
Example Results:
After evaluation, the model may yield results like this:
A
● ccuracy: 80%
● Precision: 75% (Out of all predicted rainy days, 75%were actual rainy days).
● Recall: 80% (Out of all actual rainy days, 80% werecorrectly predicted).
8. Interpreting the Model
● T he SVM model may tell you that weather patterns withhigh humidity and low pressureare
more likely to result in rain.
● It may also show that conditions likehigher windspeedandhigher pressuretend to correlate
withno rain.
Conclusion:
In this rainfall prediction scenario,SVMwith theRBF kernelcan effectively predict whether it willrain
or not based on the input weather features. By transforming the data into a higher-dimensional space,
SVM can create a robust decision boundary that maximizes the margin between rainy and non-rainy
days, providing accurate predictions for future weather conditions.
his approach can be extended to larger and more complex datasets, where SVM's ability to handle
T
non-linear relationships is crucial for making accurate predictions.
You might also like
- Improve Model Accuracy With Data Pre-ProcessingNo ratings yetImprove Model Accuracy With Data Pre-Processing11 pages
- 7 Data Preprocessing Steps in Machine LearningNo ratings yet7 Data Preprocessing Steps in Machine Learning5 pages
- Module 2-b Prediction Methods and Models-Data PreperationNo ratings yetModule 2-b Prediction Methods and Models-Data Preperation26 pages
- 01 Apply Data Preprocessing On Heart Dataset and Evaluate Performance Using Confusion MatrixNo ratings yet01 Apply Data Preprocessing On Heart Dataset and Evaluate Performance Using Confusion Matrix19 pages
- Data Preprocessing: Clean, Transform, IntegrateNo ratings yetData Preprocessing: Clean, Transform, Integrate6 pages
- 6-Deep Networks Basics - Shallow Neural Networks-29-07-2024No ratings yet6-Deep Networks Basics - Shallow Neural Networks-29-07-20248 pages
- Data Preprocessing Before Classification: Presented byNo ratings yetData Preprocessing Before Classification: Presented by23 pages
- Supervised Learning Research Paper Final With ImagesNo ratings yetSupervised Learning Research Paper Final With Images11 pages
- 1725892639module 3 The Machine Learning ProcessNo ratings yet1725892639module 3 The Machine Learning Process17 pages
- Subject - Machine Learning Group - E27-24 NameNo ratings yetSubject - Machine Learning Group - E27-24 Name18 pages
- Session-2-CO3-Introduction To Data PreprocessingNo ratings yetSession-2-CO3-Introduction To Data Preprocessing39 pages
- Supervised Learning Research Paper With ImagesNo ratings yetSupervised Learning Research Paper With Images10 pages
- Data Preprocessing: Essential Steps For Preparing Data Before ModelingNo ratings yetData Preprocessing: Essential Steps For Preparing Data Before Modeling111 pages
- Data Preprocessing and Feature EngineeringNo ratings yetData Preprocessing and Feature Engineering32 pages
- AI Based Chatbot Using NLP To Facilitate Software Development ProcessNo ratings yetAI Based Chatbot Using NLP To Facilitate Software Development Process26 pages
- Classification Basedon Decision Tree AlgorithmNo ratings yetClassification Basedon Decision Tree Algorithm10 pages
- Advanced Probability and Statistics ConceptsNo ratings yetAdvanced Probability and Statistics Concepts465 pages
- Lahann Et Al - 2019 - Utilizing Machine Learning Techniques To Reveal VAT Compliance Violations inNo ratings yetLahann Et Al - 2019 - Utilizing Machine Learning Techniques To Reveal VAT Compliance Violations in10 pages
- Business Analytics: Advance: Simple & Multiple Linear RegressionNo ratings yetBusiness Analytics: Advance: Simple & Multiple Linear Regression38 pages
- g8m6l9 - Summarizing Bivariate Data in A 2 Way TableNo ratings yetg8m6l9 - Summarizing Bivariate Data in A 2 Way Table7 pages
- Module-1-Q1C-Descriptive Statistics I - Notation50% (2)Module-1-Q1C-Descriptive Statistics I - Notation4 pages
- SDO Navotas PracticalResearch2 SHS 1stsem - FVNo ratings yetSDO Navotas PracticalResearch2 SHS 1stsem - FV100 pages
- Sample Size Determination For Survey Research and Non-Probability Sampling Techniques: A Review and Set of RecommendationsNo ratings yetSample Size Determination For Survey Research and Non-Probability Sampling Techniques: A Review and Set of Recommendations22 pages
- Comparing The Yields of Organic and Conventional Agriculture 2012No ratings yetComparing The Yields of Organic and Conventional Agriculture 20126 pages
- Practical Research2 Modules 1ST QuarterNo ratings yetPractical Research2 Modules 1ST Quarter104 pages