Linear and Logistic Regression

Machine Learning is a subset of AI, which have the ability to learn from the data trained to it and make predictions from that data. Machine learning contains a set of algorithms that work on a huge amount of data. Data is fed to algorithms to train , and on the basis of training, they build the model & perform a specific task. There are three types of machine learning supervised, unsupervised, and reinforcement learning.

Regression

Regression is a common use for supervised machine learning models. It can be defined as a predictive modeling technique which investigates the relationship between dependent and independent variables. In simple terms we can also define regression as the task of predicting a continuous quantity. Linear and Logistic regression are the most basic form of regression which are commonly used. Solving regression problems is one of the most common applications for machine learning models.

Algorithms are trained to understand the relationship between independent variables and an outcome or dependent variable. The model can then be leveraged to predict the outcome of new and unseen input data, or to fill a gap in missing data. Machine learning regression models need to understand the relationship between features and outcome variables, so accurately labelled training data is vital.

Regression analysis is used to understand the relationship between different independent variables and a dependent variable or outcome. Models that are trained to forecast or predict trends and outcomes will be trained using regression techniques. These models will learn the relationship between input and output data from labelled training data.

Linear Regression

In linear regression the data is modeled using a straight line. It can be used for the cases where we want to predict some continuous quantity. Let us find more about the algorithm. We can consider Y is having dependent variables and X is having independent variables. The response whose value need to be predicted (Y) is called as dependent variable. And the predictor value used to predict the response of the variable is called as independent variable(X).

Linear relationship between dependent and independent variables is required. For example using linear regression we can predict price of a house based on no.of bedrooms, no.of bathrooms and square foot.

If the data variable in x-axis is increasing and there is an increase in y-axis then it is known as positive linear regression. If the data variable in x-axis is increasing and there is a decrease in y-axis then it is known as negative linear regression.

Positive Correlation :

Negative Correlation :

y=mx+c is used to the model linear regression. We can calculate the linear regression using equation y=mx+c , where the value of m is:

m = 𝚺(x-x’)(y-y’)/𝚺(x-x’)2

Types of linear regression :

1. Simple Linear Regression

2. Multiple Linear Regression

1. Simple Linear Regression

Simple Linear Regression examines the relationship between the dependent variable and a single independent variable. In SLR we have one dependent(X) variable and one independent(y) variable. The equation of simple linear regression looks like : y=

2. Multiple Linear Regression

Multiple Linear Regression examines the relationship between the dependent variable and multiple independent variables. In MLR we have two or more dependent(X) variables and one independent(y) variable. The equation of simple linear regression looks like :

Logistic Regression:

Logistic regression is used to estimate the relationship between a dependent variable and one or more independent variables, but it is used to make a prediction about a categorical variable versus a continuous one. In logistic regression the data is modeled using a curve. Linear relationship between dependent and independent variables is not required. For example using logistic regression we can predict whether a patient have risk of sepsis or not.

Let us find more about the algorithm. We can consider Y is having dependent variables and X is having independent variables. The dependent variable (Y) have the response binary variable holding values like 0 or 1, yes or no, A, B , or C. And the predictor value used to predict the response of the variable is called as independent variable(X). The equation for model logistic regression is :

log [Y/( Y-Y’) ]= c+m1x1+m2x2…..

Types of logistic regression:

1. Binary

2. Multinominal

3. Ordinal

Lets go through the different types of logistic regression:

1. Binary logistic regression :

In binary type we can expect the output with two possibilities. Like yes or no. 0 or 1. For example a patient have a risk of sepsis shock or not.

2. Multinominal logistic regression :

In Multinominal type we can expect the output with three or more possibilities from the predefined classes. For example we can predict a patients health risk based on the family history.

3. Ordinal logistic regression :

In Ordinal type we can expect the output with three or more possibilities from the predefined classes and the classes are ordered. For example predicting a restaurant rating from the scale 0 to 5.

Linear vs Logistic Regression:

Linear Regression	Logistic Regression
Used for solving regression problems	Used for solving classification problems
It is used to predicting continuous dependent variable with the help of independent variables.	It is used to predict the categorical dependent variable with the help of independent variables.
The output for Linear regression should only be the continuous values​	The output of Logistic regression can only be 0 or 1 , true or false, yes or no etc.,
In linear regression, fit line is used to predict the output.	In Logistic Regression, S-curve is used to classify the samples.
Real example using linear regression we can predict price of a house based on no.of bedrooms, no.of bathrooms and square foot.	Real example using logistic regression we can predict whether a patient have risk of sepsis or not.

Conclusion :

Linear and Logistic regression algorithms are widely used by professionals to understand data in their particular field and make predictions. Linear regression and Logistic regression are both methods for modeling relationships between variables. They are both used to build statistical models but perform different tasks. Linear regression is used to model linear relationships, while logistic regression is used to model binary outcomes.