Heading image

Prerequisites

  1. Familiarity Conda package, dependency and virtual environment manager. A handy additional reference for Conda is the blog post “The Definitive Guide to Conda Environments” on “Towards Data Science”.
  2. Familiarity with JupyterLab. See here for my post on JupyterLab.
  3. These projects will also run Python notebooks on VSCode with the Jupyter Notebooks extension. If you do not use VSCode, it is expected that you know how to run notebooks (or alter the method for what works best for you).

Getting started

Let’s first clone the code from part three into the regression-with-scikit-learn-part-four directory.

What is Regularized Regression?

“Regularization” is a method to give a penalty to the model in order to prevent overfitting. The penalty is a function of the model’s complexity. The more complex the model, the higher the penalty.

  1. Ridge Regression
  2. Lasso Regression

Ridge Regression

Ridge regression tunes a model that is used to analyze data that has multicollinearity.

Lasso Regression

Lasso regression is a type of linear regression that uses shrinkage. Shrinkage is where data values are shrunk towards a central point, like the mean. This type is very useful when you have high levels of muticollinearity or when you want to automate certain parts of model selection, like variable selection/parameter elimination.

Lasso for feature selection

One of the important aspects of Lasso regression is using it to select important features of a dataset.

Lasso coefficients assigned to features

Summary

Today’s post looked into both Ridge and Lasso regression, as well as how to apply those methods using Scikit Learn.

Resources and further reading

  1. Conda
  2. JupyterLab
  3. Jupyter Notebooks
  4. “The Definitive Guide to Conda Environments”
  5. okeeffed/regression-with-scikit-learn-part-four
  6. Multicollinearity — Wikipedia
  7. Regularized Regression — statisticshowto.com

--

--

--

Senior Engineer @ UsabilityHub. Formerly Culture Amp.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Airbnb Smart Pricing: Is It Worth It?

Weekend Project: Scraping Web Data with Python and Visualizing it with PowerBI

Intuitive decision making

Create a heatmap from the logs of your activity tracker

What’s in a name?

Creating Context By Tagging Wine Tastings

Three things you should know as an Airbnb host in Seattle

Analytics Case Study: Step by Step Visualization Centric

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Dennis O'Keeffe

Dennis O'Keeffe

Senior Engineer @ UsabilityHub. Formerly Culture Amp.

More from Medium

Creating a Python Virtual Environment with Anaconda Distribution

The stupidest pandas dataframe filter

Don’t Buy Alexa Create Your Own Using Python -less than 50 Lines of Code

Alexa from Amazon

Linear Regression from scratch in Python