Scikit-Optimize-for-Hyperparameter-Tuning-in-Machine-Learning.jpg

Scikit-Optimize for Hyperparameter Tuning in Machine Learning

Hyperparameter optimization refers to performing a search in order to discover the set of specific model configuration arguments that result in the best performance of the model on a specific dataset.

There are many ways to perform hyperparameter optimization, although modern methods, such as Bayesian Optimization, are fast and effective. The Scikit-Optimize library is an open-source Python library that provides an implementation of Bayesian Optimization that can be used to tune the hyperparameters of machine learning models from the scikit-Learn Python library.

You can easily use the Scikit-Optimize library to tune the models on your next machine learning project.

In this tutorial, you will discover how to use the Scikit-Optimize library to use Bayesian Optimization for hyperparameter tuning.

After completing this tutorial, you will know:

  • Scikit-Optimize provides a general toolkit for Bayesian Optimization that can be used for hyperparameter tuning.
  • How to manually use the Scikit-Optimize library to tune the hyperparameters of a machine learning model.
  • How to use the built-in BayesSearchCV class to perform model hyperparameter tuning.

Let’s get started.

Scikit-Optimize for Hyperparameter Tuning in Machine Learning

Scikit-Optimize for Hyperparameter Tuning in Machine Learning
Photo by Dan Nevill, some rights reserved.

Tutorial Overview

This tutorial is divided into four parts; they are:

  • Scikit-Optimize
  • Machine Learning Dataset and Model
  • Manually Tune Algorithm Hyperparameters
  • Automatically Tune Algorithm Hyperparameters
  • Scikit-Optimize

    Scikit-Optimize, or skopt for short, is an open-source Python library for performing optimization tasks.

    It offers efficient optimization algorithms, such as Bayesian Optimization, and can be used to find the minimum or maximum of arbitrary cost functions.

    Bayesian Optimization provides a principled technique based on Bayes Theorem to direct a search of a global optimization problem that is efficient and effective. It works by building a probabilistic model of the objective function, called the surrogate function, that is then searched efficiently with an acquisition function before candidate samples are chosen for evaluation on the real objective function.

    For more on the topic of Bayesian Optimization, see the tutorial:

    Importantly, the library provides support for tuning the hyperparameters of machine learning algorithms offered by the scikit-learn library, so-called hyperparameter optimization. As such, it offers an efficient alternative to less efficient hyperparameter optimization procedures such as grid search and random search.

    The scikit-optimize library can be installed using pip, as follows:


    Once installed, we can import the library and print the version number to confirm the library was installed successfully and can be accessed.

    The complete example is listed below.


    Running the example reports the currently installed version number of scikit-optimize.

    Your version number should be the same or higher.


    For more installation instructions, see the documentation:

    Now that we are familiar with what Scikit-Optimize is and how to install it, let’s explore how we can use it to tune the hyperparameters of a machine learning model.

    Machine Learning Dataset and Model

    First, let’s select a standard dataset and a model to address it.

    We will use the ionosphere machine learning dataset. This is a standard machine learning dataset comprising 351 rows of data with three numerical input variables and a target variable with two class values, e.g. binary classification.

    Using a test harness of repeated stratified 10-fold cross-validation with three repeats, a naive model can achieve an accuracy of about 64 percent. A top performing model can achieve accuracy on this same test harness of about 94 percent. This provides the bounds of expected performance on this dataset.

    The dataset involves predicting whether measurements of the ionosphere indicate a specific structure or not.

    You can learn more about the dataset here:

    No need to download the dataset; we will download it automatically as part of our worked examples.

    The example below downloads the dataset and summarizes its shape.


    Running the example downloads the dataset and splits it into input and output elements. As expected, we can see that there are 351 rows of data with 34 input variables.


    We can evaluate a support vector machine (SVM) model on this dataset using repeated stratified cross-validation.

    We can report the mean model performance on the dataset averaged over all folds and repeats, which will provide a reference for model hyperparameter tuning performed in later sections.

    The complete example is listed below.


    Running the example first loads and prepares the dataset, then evaluates the SVM model on the dataset.

    Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

    In this case, we can see that the SVM with default hyperparameters achieved a mean classification accuracy of about 83.7 percent, which is skillful and close to the top performance on the problem of 94 percent.


    Next, let’s see if we can improve performance by tuning the model hyperparameters using the scikit-optimize library.

    Manually Tune Algorithm Hyperparameters

    The Scikit-Optimize library can be used to tune the hyperparameters of a machine learning model.

    We can achieve this manually by using the Bayesian Optimization capabilities of the library.

    This requires that we first define a search space. In this case, this will be the hyperparameters of the model that we wish to tune, and the scope or range of each hyperparameter.

    We will tune the following hyperparameters of the SVM model:

    • C, the regularization parameter.
    • kernel, the type of kernel used in the model.
    • degree, used for the polynomial kernel.
    • gamma, used in most other kernels.

    For the numeric hyperparameters C and gamma, we will define a log scale to search between a small value of 1e-6 and 100. Degree is an integer and we will search values between 1 and 5. Finally, the kernel is a categorical variable with specific named values.

    We can define the search space for these four hyperparameters, a list of data types from the skopt library, as follows:


    Note the data type, the range, and the name of the hyperparameter specified for each.

    We can then define a function that will be called by the search procedure. This is a function expected by the optimization procedure later and takes a model and set of specific hyperparameters for the model, evaluates it, and returns a score for the set of hyperparameters.

    In our case, we want to evaluate the model using repeated stratified 10-fold cross-validation on our ionosphere dataset. We want to maximize classification accuracy, e.g. find the set of model hyperparameters that give the best accuracy. By default, the process minimizes the score returned from this function, therefore, we will return one minus the accuracy, e.g. perfect skill will be (1 – accuracy) or 0.0, and the worst skill will be 1.0.

    The evaluate_model() function below implements this and takes a specific set of hyperparameters.


    Next, we can execute the search by calling the gp_minimize() function and passing the name of the function to call to evaluate each model and the search space to optimize.


    The procedure will run until it converges and returns a result.

    The result object contains lots of details, but importantly, we can access the score of the best performing configuration and the hyperparameters used by the best forming model.


    Tying this together, the complete example of manually tuning the hyperparameters of an SVM on the ionosphere dataset is listed below.


    Running the example may take a few moments, depending on the speed of your machine.

    You may see some warning messages that you can safely ignore, such as:


    At the end of the run, the best-performing configuration is reported.

    Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

    In this case, we can see that configuration, reported in order of the search space list, was a modest C value, a RBF kernel, a degree of 2 (ignored by the RBF kernel), and a modest gamma value.

    Importantly, we can see that the skill of this model was approximately 94.7 percent, which is a top-performing model


    This is not the only way to use the Scikit-Optimize library for hyperparameter tuning. In the next section, we can see a more automated approach.

    Automatically Tune Algorithm Hyperparameters

    The Scikit-Learn machine learning library provides tools for tuning model hyperparameters.

    Specifically, it provides the GridSearchCV and RandomizedSearchCV classes that take a model, a search space, and a cross-validation configuration.

    The benefit of these classes is that the search procedure is performed automatically, requiring minimal configuration.

    Similarly, the Scikit-Optimize library provides a similar interface for performing a Bayesian Optimization of model hyperparameters via the BayesSearchCV class.

    This class can be used in the same way as the Scikit-Learn equivalents.

    First, the search space must be defined as a dictionary with hyperparameter names used as the key and the scope of the variable as the value.


    We can then define the BayesSearchCV configuration taking the model we wish to evaluate, the hyperparameter search space, and the cross-validation configuration.


    We can then execute the search and report the best result and configuration at the end.


    Tying this together, the complete example of automatically tuning SVM hyperparameters using the BayesSearchCV class on the ionosphere dataset is listed below.


    Running the example may take a few moments, depending on the speed of your machine.

    You may see some warning messages that you can safely ignore, such as:


    At the end of the run, the best-performing configuration is reported.

    Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

    In this case, we can see that the model performed above top-performing models achieving a mean classification accuracy of about 95.2 percent.

    The search discovered a large C value, an RBF kernel, and a small gamma value.


    This provides a template that you can use to tune the hyperparameters on your machine learning project.

    Further Reading

    This section provides more resources on the topic if you are looking to go deeper.

    Related Tutorials
    APIs

    Summary

    In this tutorial, you discovered how to use the Scikit-Optimize library to use Bayesian Optimization for hyperparameter tuning.

    Specifically, you learned:

    • Scikit-Optimize provides a general toolkit for Bayesian Optimization that can be used for hyperparameter tuning.
    • How to manually use the Scikit-Optimize library to tune the hyperparameters of a machine learning model.
    • How to use the built-in BayesSearchCV class to perform model hyperparameter tuning.

    Do you have any questions?
    Ask your questions in the comments below and I will do my best to answer.

    Discover Fast Machine Learning in Python!

    Master Machine Learning With Python
    Develop Your Own Models in Minutes

    …with just a few lines of scikit-learn code

    Learn how in my new Ebook:
    Machine Learning Mastery With Python

    Covers self-study tutorials and end-to-end projects like:
    Loading data, visualization, modeling, tuning, and much more…

    Finally Bring Machine Learning To

    Your Own Projects

    Skip the Academics. Just Results.

    See What’s Inside

    Covid Abruzzo Basilicata Calabria Campania Emilia Romagna Friuli Venezia Giulia Lazio Liguria Lombardia Marche Molise Piemonte Puglia Sardegna Sicilia Toscana Trentino Alto Adige Umbria Valle d’Aosta Veneto Italia Agrigento Alessandria Ancona Aosta Arezzo Ascoli Piceno Asti Avellino Bari Barletta-Andria-Trani Belluno Benevento Bergamo Biella Bologna Bolzano Brescia Brindisi Cagliari Caltanissetta Campobasso Carbonia-Iglesias Caserta Catania Catanzaro Chieti Como Cosenza Cremona Crotone Cuneo Enna Fermo Ferrara Firenze Foggia Forlì-Cesena Frosinone Genova Gorizia Grosseto Imperia Isernia La Spezia L’Aquila Latina Lecce Lecco Livorno Lodi Lucca Macerata Mantova Massa-Carrara Matera Messina Milano Modena Monza e della Brianza Napoli Novara Nuoro Olbia-Tempio Oristano Padova Palermo Parma Pavia Perugia Pesaro e Urbino Pescara Piacenza Pisa Pistoia Pordenone Potenza Prato Ragusa Ravenna Reggio Calabria Reggio Emilia Rieti Rimini Roma Rovigo Salerno Medio Campidano Sassari Savona Siena Siracusa Sondrio Taranto Teramo Terni Torino Ogliastra Trapani Trento Treviso Trieste Udine Varese Venezia Verbano-Cusio-Ossola Vercelli Verona Vibo Valentia Vicenza Viterbo

    1598716808_524_Plot-a-Decision-Surface-for-Machine-Learning-Algorithms-in-Python.png

    Plot a Decision Surface for Machine Learning Algorithms in Python

    Last Updated on August 26, 2020

    Classification algorithms learn how to assign class labels to examples, although their decisions can appear opaque.

    A popular diagnostic for understanding the decisions made by a classification algorithm is the decision surface. This is a plot that shows how a fit machine learning algorithm predicts a coarse grid across the input feature space.

    A decision surface plot is a powerful tool for understanding how a given model “sees” the prediction task and how it has decided to divide the input feature space by class label.

    In this tutorial, you will discover how to plot a decision surface for a classification machine learning algorithm.

    After completing this tutorial, you will know:

    • Decision surface is a diagnostic tool for understanding how a classification algorithm divides up the feature space.
    • How to plot a decision surface for using crisp class labels for a machine learning algorithm.
    • How to plot and interpret a decision surface using predicted probabilities.

    Kick-start your project with my new book Machine Learning Mastery With Python, including step-by-step tutorials and the Python source code files for all examples.

    Let’s get started.

    Plot a Decision Surface for Machine Learning Algorithms in Python

    Plot a Decision Surface for Machine Learning Algorithms in Python
    Photo by Tony Webster, some rights reserved.

    Tutorial Overview

    This tutorial is divided into three parts; they are:

  • Decision Surface
  • Dataset and Model
  • Plot a Decision Surface
  • Decision Surface

    Classification machine learning algorithms learn to assign labels to input examples.

    Consider numeric input features for the classification task defining a continuous input feature space.

    We can think of each input feature defining an axis or dimension on a feature space. Two input features would define a feature space that is a plane, with dots representing input coordinates in the input space. If there were three input variables, the feature space would be a three-dimensional volume.

    Each point in the space can be assigned a class label. In terms of a two-dimensional feature space, we can think of each point on the planing having a different color, according to their assigned class.

    The goal of a classification algorithm is to learn how to divide up the feature space such that labels are assigned correctly to points in the feature space, or at least, as correctly as is possible.

    This is a useful geometric understanding of classification predictive modeling. We can take it one step further.

    Once a classification machine learning algorithm divides a feature space, we can then classify each point in the feature space, on some arbitrary grid, to get an idea of how exactly the algorithm chose to divide up the feature space.

    This is called a decision surface or decision boundary, and it provides a diagnostic tool for understanding a model on a classification predictive modeling task.

    Although the notion of a “surface” suggests a two-dimensional feature space, the method can be used with feature spaces with more than two dimensions, where a surface is created for each pair of input features.

    Now that we are familiar with what a decision surface is, next, let’s define a dataset and model for which we later explore the decision surface.

    Dataset and Model

    In this section, we will define a classification task and predictive model to learn the task.

    Synthetic Classification Dataset

    We can use the make_blobs() scikit-learn function to define a classification task with a two-dimensional class numerical feature space and each point assigned one of two class labels, e.g. a binary classification task.


    Once defined, we can then create a scatter plot of the feature space with the first feature defining the x-axis, the second feature defining the y axis, and each sample represented as a point in the feature space.

    We can then color points in the scatter plot according to their class label as either 0 or 1.


    Tying this together, the complete example of defining and plotting a synthetic classification dataset is listed below.


    Running the example creates the dataset, then plots the dataset as a scatter plot with points colored by class label.

    We can see a clear separation between examples from the two classes and we can imagine how a machine learning model might draw a line to separate the two classes, e.g. perhaps a diagonal line right through the middle of the two groups.

    Scatter Plot of Binary Classification Dataset With 2D Feature Space

    Scatter Plot of Binary Classification Dataset With 2D Feature Space

    Fit Classification Predictive Model

    We can now fit a model on our dataset.

    In this case, we will fit a logistic regression algorithm because we can predict both crisp class labels and probabilities, both of which we can use in our decision surface.

    We can define the model, then fit it on the training dataset.


    Once defined, we can use the model to make a prediction for the training dataset to get an idea of how well it learned to divide the feature space of the training dataset and assign labels.


    The predictions can be evaluated using classification accuracy.


    Tying this together, the complete example of fitting and evaluating a model on the synthetic binary classification dataset is listed below.


    Running the example fits the model and makes a prediction for each example.

    Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

    In this case, we can see that the model achieved a performance of about 97.2 percent.


    Now that we have a dataset and model, let’s explore how we can develop a decision surface.

    Plot a Decision Surface

    We can create a decision surface by fitting a model on the training dataset, then using the model to make predictions for a grid of values across the input domain.

    Once we have the grid of predictions, we can plot the values and their class label.

    A scatter plot could be used if a fine enough grid was taken. A better approach is to use a contour plot that can interpolate the colors between the points.

    The contourf() Matplotlib function can be used.

    This requires a few steps.

    First, we need to define a grid of points across the feature space.

    To do this, we can find the minimum and maximum values for each feature and expand the grid one step beyond that to ensure the whole feature space is covered.


    We can then create a uniform sample across each dimension using the arange() function at a chosen resolution. We will use a resolution of 0.1 in this case.


    Now we need to turn this into a grid.

    We can use the meshgrid() NumPy function to create a grid from these two vectors.

    If the first feature x1 is our x-axis of the feature space, then we need one row of x1 values of the grid for each point on the y-axis.

    Similarly, if we take x2 as our y-axis of the feature space, then we need one column of x2 values of the grid for each point on the x-axis.

    The meshgrid() function will do this for us, duplicating the rows and columns for us as needed. It returns two grids for the two input vectors. The first grid of x-values and the second of y-values, organized in an appropriately sized grid of rows and columns across the feature space.


    We then need to flatten out the grid to create samples that we can feed into the model and make a prediction.

    To do this, first, we flatten each grid into a vector.


    Then we stack the vectors side by side as columns in an input dataset, e.g. like our original training dataset, but at a much higher resolution.


    We can then feed this into our model and get a prediction for each point in the grid.


    So far, so good.

    We have a grid of values across the feature space and the class labels as predicted by our model.

    Next, we need to plot the grid of values as a contour plot.

    The contourf() function takes separate grids for each axis, just like what was returned from our prior call to meshgrid(). Great!

    So we can use xx and yy that we prepared earlier and simply reshape the predictions (yhat) from the model to have the same shape.


    We then plot the decision surface with a two-color colormap.


    We can then plot the actual points of the dataset over the top to see how well they were separated by the logistic regression decision surface.

    The complete example of plotting a decision surface for a logistic regression model on our synthetic binary classification dataset is listed below.


    Running the example fits the model and uses it to predict outcomes for the grid of values across the feature space and plots the result as a contour plot.

    We can see, as we might have suspected, logistic regression divides the feature space using a straight line. It is a linear model, after all; this is all it can do.

    Creating a decision surface is almost like magic. It gives immediate and meaningful insight into how the model has learned the task.

    Try it with different algorithms, like an SVM or decision tree.
    Post your resulting maps as links in the comments below!

    Decision Surface for Logistic Regression on a Binary Classification Task

    Decision Surface for Logistic Regression on a Binary Classification Task

    We can add more depth to the decision surface by using the model to predict probabilities instead of class labels.


    When plotted, we can see how confident or likely it is that each point in the feature space belongs to each of the class labels, as seen by the model.

    We can use a different color map that has gradations, and show a legend so we can interpret the colors.


    The complete example of creating a decision surface using probabilities is listed below.


    Running the example predicts the probability of class membership for each point on the grid across the feature space and plots the result.

    Here, we can see that the model is unsure (lighter colors) around the middle of the domain, given the sampling noise in that area of the feature space. We can also see that the model is very confident (full colors) in the bottom-left and top-right halves of the domain.

    Together, the crisp class and probability decision surfaces are powerful diagnostic tools for understanding your model and how it divides the feature space for your predictive modeling task.

    Probability Decision Surface for Logistic Regression on a Binary Classification Task

    Probability Decision Surface for Logistic Regression on a Binary Classification Task

    Further Reading

    This section provides more resources on the topic if you are looking to go deeper.

    Summary

    In this tutorial, you discovered how to plot a decision surface for a classification machine learning algorithm.

    Specifically, you learned:

    • Decision surface is a diagnostic tool for understanding how a classification algorithm divides up the feature space.
    • How to plot a decision surface for using crisp class labels for a machine learning algorithm.
    • How to plot and interpret a decision surface using predicted probabilities.

    Do you have any questions?
    Ask your questions in the comments below and I will do my best to answer.

    Discover Fast Machine Learning in Python!

    Master Machine Learning With Python
    Develop Your Own Models in Minutes

    …with just a few lines of scikit-learn code

    Learn how in my new Ebook:
    Machine Learning Mastery With Python

    Covers self-study tutorials and end-to-end projects like:
    Loading data, visualization, modeling, tuning, and much more…

    Finally Bring Machine Learning To

    Your Own Projects

    Skip the Academics. Just Results.

    See What’s Inside

    Covid Abruzzo Basilicata Calabria Campania Emilia Romagna Friuli Venezia Giulia Lazio Liguria Lombardia Marche Molise Piemonte Puglia Sardegna Sicilia Toscana Trentino Alto Adige Umbria Valle d’Aosta Veneto Italia Agrigento Alessandria Ancona Aosta Arezzo Ascoli Piceno Asti Avellino Bari Barletta-Andria-Trani Belluno Benevento Bergamo Biella Bologna Bolzano Brescia Brindisi Cagliari Caltanissetta Campobasso Carbonia-Iglesias Caserta Catania Catanzaro Chieti Como Cosenza Cremona Crotone Cuneo Enna Fermo Ferrara Firenze Foggia Forlì-Cesena Frosinone Genova Gorizia Grosseto Imperia Isernia La Spezia L’Aquila Latina Lecce Lecco Livorno Lodi Lucca Macerata Mantova Massa-Carrara Matera Messina Milano Modena Monza e della Brianza Napoli Novara Nuoro Olbia-Tempio Oristano Padova Palermo Parma Pavia Perugia Pesaro e Urbino Pescara Piacenza Pisa Pistoia Pordenone Potenza Prato Ragusa Ravenna Reggio Calabria Reggio Emilia Rieti Rimini Roma Rovigo Salerno Medio Campidano Sassari Savona Siena Siracusa Sondrio Taranto Teramo Terni Torino Ogliastra Trapani Trento Treviso Trieste Udine Varese Venezia Verbano-Cusio-Ossola Vercelli Verona Vibo Valentia Vicenza Viterbo

    1598763608_908_How-to-use-Seaborn-Data-Visualization-for-Machine-Learning.png

    How to use Seaborn Data Visualization for Machine Learning

    Last Updated on August 19, 2020

    Data visualization provides insight into the distribution and relationships between variables in a dataset.

    This insight can be helpful in selecting data preparation techniques to apply prior to modeling and the types of algorithms that may be most suited to the data.

    Seaborn is a data visualization library for Python that runs on top of the popular Matplotlib data visualization library, although it provides a simple interface and aesthetically better-looking plots.

    In this tutorial, you will discover a gentle introduction to Seaborn data visualization for machine learning.

    After completing this tutorial, you will know:

    • How to summarize the distribution of variables using bar charts, histograms, and box and whisker plots.
    • How to summarize relationships using line plots and scatter plots.
    • How to compare the distribution and relationships of variables for different class values on the same plot.

    Kick-start your project with my new book Machine Learning Mastery With Python, including step-by-step tutorials and the Python source code files for all examples.

    Let’s get started.

    How to use Seaborn Data Visualization for Machine Learning

    How to use Seaborn Data Visualization for Machine Learning
    Photo by Martin Pettitt, some rights reserved.

    Tutorial Overview

    This tutorial is divided into six parts; they are:

    • Seaborn Data Visualization Library
    • Line Plots
    • Bar Chart Plots
    • Histogram Plots
    • Box and Whisker Plots
    • Scatter Plots

    Seaborn Data Visualization Library

    The primary plotting library for Python is called Matplotlib.

    Seaborn is a plotting library that offers a simpler interface, sensible defaults for plots needed for machine learning, and most importantly, the plots are aesthetically better looking than those in Matplotlib.

    Seaborn requires that Matplotlib is installed first.

    You can install Matplotlib directly using pip, as follows:


    Once installed, you can confirm that the library can be loaded and used by printing the version number, as follows:


    Running the example prints the current version of the Matplotlib library.


    Next, the Seaborn library can be installed, also using pip:


    Once installed, we can also confirm the library can be loaded and used by printing the version number, as follows:


    Running the example prints the current version of the Seaborn library.


    To create Seaborn plots, you must import the Seaborn library and call functions to create the plots.

    Importantly, Seaborn plotting functions expect data to be provided as Pandas DataFrames. This means that if you are loading your data from CSV files, you must use Pandas functions like read_csv() to load your data as a DataFrame. When plotting, columns can then be specified via the DataFrame name or column index.

    To show the plot, you can call the show() function on Matplotlib library.


    Alternatively, the plots can be saved to file, such as a PNG formatted image file. The savefig() Matplotlib function can be used to save images.


    Now that we have Seaborn installed, let’s look at some common plots we may need when working with machine learning data.

    Line Plots

    A line plot is generally used to present observations collected at regular intervals.

    The x-axis represents the regular interval, such as time. The y-axis shows the observations, ordered by the x-axis and connected by a line.

    A line plot can be created in Seaborn by calling the lineplot() function and passing the x-axis data for the regular interval, and y-axis for the observations.

    We can demonstrate a line plot using a time series dataset of monthly car sales.

    The dataset has two columns: “Month” and “Sales.” Month will be used as the x-axis and Sales will be plotted on the y-axis.


    Tying this together, the complete example is listed below.


    Running the example first loads the time series dataset and creates a line plot of the data, clearly showing a trend and seasonality in the sales data.

    Line Plot of a Time Series Dataset

    Line Plot of a Time Series Dataset

    For more great examples of line plots with Seaborn, see: Visualizing statistical relationships.

    Bar Chart Plots

    A bar chart is generally used to present relative quantities for multiple categories.

    The x-axis represents the categories that are spaced evenly. The y-axis represents the quantity for each category and is drawn as a bar from the baseline to the appropriate level on the y-axis.

    A bar chart can be created in Seaborn by calling the countplot() function and passing the data.

    We will demonstrate a bar chart with a variable from the breast cancer classification dataset that is comprised of categorical input variables.

    We will just plot one variable, in this case, the first variable which is the age bracket.


    Tying this together, the complete example is listed below.


    Running the example first loads the breast cancer dataset and creates a bar chart plot of the data, showing each age group and the number of individuals (samples) that fall within reach group.

    Bar Chart Plot of Age Range Categorical Variable

    Bar Chart Plot of Age Range Categorical Variable

    We might also want to plot the counts for each category for a variable, such as the first variable, against the class label.

    This can be achieved using the countplot() function and specifying the class variable (column index 9) via the “hue” argument, as follows:


    Tying this together, the complete example is listed below.


    Running the example first loads the breast cancer dataset and creates a bar chart plot of the data, showing each age group and the number of individuals (samples) that fall within each group separated by the two class labels for the dataset.

    Bar Chart Plot of Age Range Categorical Variable by Class Label

    Bar Chart Plot of Age Range Categorical Variable by Class Label

    For more great examples of bar chart plots with Seaborn, see: Plotting with categorical data.

    Histogram Plots

    A histogram plot is generally used to summarize the distribution of a numerical data sample.

    The x-axis represents discrete bins or intervals for the observations. For example, observations with values between 1 and 10 may be split into five bins, the values [1,2] would be allocated to the first bin, [3,4] would be allocated to the second bin, and so on.

    The y-axis represents the frequency or count of the number of observations in the dataset that belong to each bin.

    Essentially, a data sample is transformed into a bar chart where each category on the x-axis represents an interval of observation values.

    A histogram can be created in Seaborn by calling the distplot() function and passing the variable.

    We will demonstrate a boxplot with a numerical variable from the diabetes classification dataset. We will just plot one variable, in this case, the first variable, which is the number of times that a patient was pregnant.


    Tying this together, the complete example is listed below.


    Running the example first loads the diabetes dataset and creates a histogram plot of the variable, showing the distribution of the values with a hard cut-off at zero.

    The plot shows both the histogram (counts of bins) as well as a smooth estimate of the probability density function.

    Histogram Plot of Number of Times Pregnant Numerical Variable

    Histogram Plot of Number of Times Pregnant Numerical Variable

    For more great examples of histogram plots with Seaborn, see: Visualizing the distribution of a dataset.

    Box and Whisker Plots

    A box and whisker plot, or boxplot for short, is generally used to summarize the distribution of a data sample.

    The x-axis is used to represent the data sample, where multiple boxplots can be drawn side by side on the x-axis if desired.

    The y-axis represents the observation values. A box is drawn to summarize the middle 50 percent of the dataset starting at the observation at the 25th percentile and ending at the 75th percentile. This is called the interquartile range, or IQR. The median, or 50th percentile, is drawn with a line.

    Lines called whiskers are drawn extending from both ends of the box, calculated as (1.5 * IQR) to demonstrate the expected range of sensible values in the distribution. Observations outside the whiskers might be outliers and are drawn with small circles.

    A boxplot can be created in Seaborn by calling the boxplot() function and passing the data.

    We will demonstrate a boxplot with a numerical variable from the diabetes classification dataset. We will just plot one variable, in this case, the first variable, which is the number of times that a patient was pregnant.


    Tying this together, the complete example is listed below.


    Running the example first loads the diabetes dataset and creates a boxplot plot of the first input variable, showing the distribution of the number of times patients were pregnant.

    We can see the median just above 2.5 times, some outliers up around 15 times (wow!).

    Box and Whisker Plot of Number of Times Pregnant Numerical Variable

    Box and Whisker Plot of Number of Times Pregnant Numerical Variable

    We might also want to plot the distribution of the numerical variable for each value of a categorical variable, such as the first variable, against the class label.

    This can be achieved by calling the boxplot() function and passing the class variable as the x-axis and the numerical variable as the y-axis.


    Tying this together, the complete example is listed below.


    Running the example first loads the diabetes dataset and creates a boxplot of the data, showing the distribution of the number of times pregnant as a numerical variable for the two-class labels.

    Box and Whisker Plot of Number of Times Pregnant Numerical Variable by Class Label

    Box and Whisker Plot of Number of Times Pregnant Numerical Variable by Class Label

    Scatter Plots

    A scatter plot, or scatterplot, is generally used to summarize the relationship between two paired data samples.

    Paired data samples mean that two measures were recorded for a given observation, such as the weight and height of a person.

    The x-axis represents observation values for the first sample, and the y-axis represents the observation values for the second sample. Each point on the plot represents a single observation.

    A scatterplot can be created in Seaborn by calling the scatterplot() function and passing the two numerical variables.

    We will demonstrate a scatterplot with two numerical variables from the diabetes classification dataset. We will plot the first versus the second variable, in this case, the first variable, which is the number of times that a patient was pregnant, and the second is the plasma glucose concentration after a two hour oral glucose tolerance test (more details of the variables here).


    Tying this together, the complete example is listed below.


    Running the example first loads the diabetes dataset and creates a scatter plot of the first two input variables.

    We can see a somewhat uniform relationship between the two variables.

    Scatter Plot of Number of Times Pregnant vs. Plasma Glucose Numerical Variables

    Scatter Plot of Number of Times Pregnant vs. Plasma Glucose Numerical Variables

    We might also want to plot the relationship for the pair of numerical variables against the class label.

    This can be achieved using the scatterplot() function and specifying the class variable (column index 8) via the “hue” argument, as follows:


    Tying this together, the complete example is listed below.


    Running the example first loads the diabetes dataset and creates a scatter plot of the first two variables vs. class label.

    Scatter Plot of Number of Times Pregnant vs. Plasma Glucose Numerical Variables by Class Label

    Scatter Plot of Number of Times Pregnant vs. Plasma Glucose Numerical Variables by Class Label

    Further Reading

    This section provides more resources on the topic if you are looking to go deeper.

    Tutorials
    APIs

    Summary

    In this tutorial, you discovered a gentle introduction to Seaborn data visualization for machine learning.

    Specifically, you learned:

    • How to summarize the distribution of variables using bar charts, histograms, and box and whisker plots.
    • How to summarize relationships using line plots and scatter plots.
    • How to compare the distribution and relationships of variables for different class values on the same plot.

    Do you have any questions?
    Ask your questions in the comments below and I will do my best to answer.

    Discover Fast Machine Learning in Python!

    Master Machine Learning With Python
    Develop Your Own Models in Minutes

    …with just a few lines of scikit-learn code

    Learn how in my new Ebook:
    Machine Learning Mastery With Python

    Covers self-study tutorials and end-to-end projects like:
    Loading data, visualization, modeling, tuning, and much more…

    Finally Bring Machine Learning To

    Your Own Projects

    Skip the Academics. Just Results.

    See What’s Inside

    Covid Abruzzo Basilicata Calabria Campania Emilia Romagna Friuli Venezia Giulia Lazio Liguria Lombardia Marche Molise Piemonte Puglia Sardegna Sicilia Toscana Trentino Alto Adige Umbria Valle d’Aosta Veneto Italia Agrigento Alessandria Ancona Aosta Arezzo Ascoli Piceno Asti Avellino Bari Barletta-Andria-Trani Belluno Benevento Bergamo Biella Bologna Bolzano Brescia Brindisi Cagliari Caltanissetta Campobasso Carbonia-Iglesias Caserta Catania Catanzaro Chieti Como Cosenza Cremona Crotone Cuneo Enna Fermo Ferrara Firenze Foggia Forlì-Cesena Frosinone Genova Gorizia Grosseto Imperia Isernia La Spezia L’Aquila Latina Lecce Lecco Livorno Lodi Lucca Macerata Mantova Massa-Carrara Matera Messina Milano Modena Monza e della Brianza Napoli Novara Nuoro Olbia-Tempio Oristano Padova Palermo Parma Pavia Perugia Pesaro e Urbino Pescara Piacenza Pisa Pistoia Pordenone Potenza Prato Ragusa Ravenna Reggio Calabria Reggio Emilia Rieti Rimini Roma Rovigo Salerno Medio Campidano Sassari Savona Siena Siracusa Sondrio Taranto Teramo Terni Torino Ogliastra Trapani Trento Treviso Trieste Udine Varese Venezia Verbano-Cusio-Ossola Vercelli Verona Vibo Valentia Vicenza Viterbo

    Recent Posts

    Archives

    wpChatIcon