How-to-Use-AutoKeras-for-Classification-and-Regression.jpg

How to Use AutoKeras for Classification and Regression

AutoML refers to techniques for automatically discovering the best-performing model for a given dataset.

When applied to neural networks, this involves both discovering the model architecture and the hyperparameters used to train the model, generally referred to as neural architecture search.

AutoKeras is an open-source library for performing AutoML for deep learning models. The search is performed using so-called Keras models via the TensorFlow tf.keras API.

It provides a simple and effective approach for automatically finding top-performing models for a wide range of predictive modeling tasks, including tabular or so-called structured classification and regression datasets.

In this tutorial, you will discover how to use AutoKeras to find good neural network models for classification and regression tasks.

After completing this tutorial, you will know:

  • AutoKeras is an implementation of AutoML for deep learning that uses neural architecture search.
  • How to use AutoKeras to find a top-performing model for a binary classification dataset.
  • How to use AutoKeras to find a top-performing model for a regression dataset.

Let’s get started.

How to Use AutoKeras for Classification and Regression

How to Use AutoKeras for Classification and Regression
Photo by kanu101, some rights reserved.

Tutorial Overview

This tutorial is divided into three parts; they are:

  • AutoKeras for Deep Learning
  • AutoKeras for Classification
  • AutoKeras for Regression
  • AutoKeras for Deep Learning

    Automated Machine Learning, or AutoML for short, refers to automatically finding the best combination of data preparation, model, and model hyperparameters for a predictive modeling problem.

    The benefit of AutoML is allowing machine learning practitioners to quickly and effectively address predictive modeling tasks with very little input, e.g. fire and forget.

    Automated Machine Learning (AutoML) has become a very important research topic with wide applications of machine learning techniques. The goal of AutoML is to enable people with limited machine learning background knowledge to use machine learning models easily.

    — Auto-keras: An efficient neural architecture search system, 2019.

    AutoKeras is an implementation of AutoML for deep learning models using the Keras API, specifically the tf.keras API provided by TensorFlow 2.

    It uses a process of searching through neural network architectures to best address a modeling task, referred to more generally as Neural Architecture Search, or NAS for short.

    … we have developed a widely adopted open-source AutoML system based on our proposed method, namely Auto-Keras. It is an open-source AutoML system, which can be downloaded and installed locally.

    — Auto-keras: An efficient neural architecture search system, 2019.

    In the spirit of Keras, AutoKeras provides an easy-to-use interface for different tasks, such as image classification, structured data classification or regression, and more. The user is only required to specify the location of the data and the number of models to try and is returned a model that achieves the best performance (under the configured constraints) on that dataset.

    Note: AutoKeras provides a TensorFlow 2 Keras model (e.g. tf.keras) and not a Standalone Keras model. As such, the library assumes that you have Python 3 and TensorFlow 2.1 or higher installed.

    To install AutoKeras, you can use Pip, as follows:


    You can confirm the installation was successful and check the version number as follows:


    You should see output like the following:


    Once installed, you can then apply AutoKeras to find a good or great neural network model for your predictive modeling task.

    We will take a look at two common examples where you may want to use AutoKeras, classification and regression on tabular data, so-called structured data.

    AutoKeras for Classification

    AutoKeras can be used to discover a good or great model for classification tasks on tabular data.

    Recall tabular data are those datasets composed of rows and columns, such as a table or data as you would see in a spreadsheet.

    In this section, we will develop a model for the Sonar classification dataset for classifying sonar returns as rocks or mines. This dataset consists of 208 rows of data with 60 input features and a target class label of 0 (rock) or 1 (mine).

    A naive model can achieve a classification accuracy of about 53.4 percent via repeated 10-fold cross-validation, which provides a lower-bound. A good model can achieve an accuracy of about 88.2 percent, providing an upper-bound.

    You can learn more about the dataset here:

    No need to download the dataset; we will download it automatically as part of the example.

    First, we can download the dataset and split it into a randomly selected train and test set, holding 33 percent for test and using 67 percent for training.

    The complete example is listed below.


    Running the example first downloads the dataset and summarizes the shape, showing the expected number of rows and columns.

    The dataset is then split into input and output elements, then these elements are further split into train and test datasets.


    We can use AutoKeras to automatically discover an effective neural network model for this dataset.

    This can be achieved by using the StructuredDataClassifier class and specifying the number of models to search. This defines the search to perform.


    We can then execute the search using our loaded dataset.


    This may take a few minutes and will report the progress of the search.

    Next, we can evaluate the model on the test dataset to see how it performs on new data.


    We then use the model to make a prediction for a new row of data.


    We can retrieve the final model, which is an instance of a TensorFlow Keras model.


    We can then summarize the structure of the model to see what was selected.


    Finally, we can save the model to file for later use, which can be loaded using the TensorFlow load_model() function.


    Tying this together, the complete example of applying AutoKeras to find an effective neural network model for the Sonar dataset is listed below.


    Running the example will report a lot of debug information about the progress of the search.

    The models and results are all saved in a folder called “structured_data_classifier” in your current working directory.


    The best-performing model is then evaluated on the hold-out test dataset.

    Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

    In this case, we can see that the model achieved a classification accuracy of about 82.6 percent.


    Next, the architecture of the best-performing model is reported.

    We can see a model with two hidden layers with dropout and ReLU activation.


    AutoKeras for Regression

    AutoKeras can also be used for regression tasks, that is, predictive modeling problems where a numeric value is predicted.

    We will use the auto insurance dataset that involves predicting the total payment from claims given the total number of claims. The dataset has 63 rows and one input and one output variable.

    A naive model can achieve a mean absolute error (MAE) of about 66 using repeated 10-fold cross-validation, providing a lower-bound on expected performance. A good model can achieve a MAE of about 28, providing a performance upper-bound.

    You can learn more about this dataset here:

    We can load the dataset and split it into input and output elements and then train and test datasets.

    The complete example is listed below.


    Running the example loads the dataset, confirming the number of rows and columns, then splits the dataset into train and test sets.


    AutoKeras can be applied to a regression task using the StructuredDataRegressor class and configured for the number of models to trial.


    The search can then be run and the best model saved, much like in the classification case.


    We can then use the best-performing model and evaluate it on the hold out dataset, make a prediction on new data, and summarize its structure.


    Tying this together, the complete example of using AutoKeras to discover an effective neural network model for the auto insurance dataset is listed below.


    Running the example will report a lot of debug information about the progress of the search.

    The models and results are all saved in a folder called “structured_data_regressor” in your current working directory.


    The best-performing model is then evaluated on the hold-out test dataset.

    Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

    In this case, we can see that the model achieved a MAE of about 24.


    Next, the architecture of the best-performing model is reported.

    We can see a model with two hidden layers with ReLU activation.


    Further Reading

    This section provides more resources on the topic if you are looking to go deeper.

    Summary

    In this tutorial, you discovered how to use AutoKeras to find good neural network models for classification and regression tasks.

    Specifically, you learned:

    • AutoKeras is an implementation of AutoML for deep learning that uses neural architecture search.
    • How to use AutoKeras to find a top-performing model for a binary classification dataset.
    • How to use AutoKeras to find a top-performing model for a regression dataset.

    Do you have any questions?
    Ask your questions in the comments below and I will do my best to answer.

    Develop Deep Learning Projects with Python!

    Deep Learning with Python
     What If You Could Develop A Network in Minutes

    …with just a few lines of Python

    Discover how in my new Ebook:
    Deep Learning With Python

    It covers end-to-end projects on topics like:
    Multilayer Perceptrons, Convolutional Nets and Recurrent Neural Nets, and more…

    Finally Bring Deep Learning To

    Your Own Projects

    Skip the Academics. Just Results.

    See What’s Inside

    Covid Abruzzo Basilicata Calabria Campania Emilia Romagna Friuli Venezia Giulia Lazio Liguria Lombardia Marche Molise Piemonte Puglia Sardegna Sicilia Toscana Trentino Alto Adige Umbria Valle d’Aosta Veneto Italia Agrigento Alessandria Ancona Aosta Arezzo Ascoli Piceno Asti Avellino Bari Barletta-Andria-Trani Belluno Benevento Bergamo Biella Bologna Bolzano Brescia Brindisi Cagliari Caltanissetta Campobasso Carbonia-Iglesias Caserta Catania Catanzaro Chieti Como Cosenza Cremona Crotone Cuneo Enna Fermo Ferrara Firenze Foggia Forlì-Cesena Frosinone Genova Gorizia Grosseto Imperia Isernia La Spezia L’Aquila Latina Lecce Lecco Livorno Lodi Lucca Macerata Mantova Massa-Carrara Matera Messina Milano Modena Monza e della Brianza Napoli Novara Nuoro Olbia-Tempio Oristano Padova Palermo Parma Pavia Perugia Pesaro e Urbino Pescara Piacenza Pisa Pistoia Pordenone Potenza Prato Ragusa Ravenna Reggio Calabria Reggio Emilia Rieti Rimini Roma Rovigo Salerno Medio Campidano Sassari Savona Siena Siracusa Sondrio Taranto Teramo Terni Torino Ogliastra Trapani Trento Treviso Trieste Udine Varese Venezia Verbano-Cusio-Ossola Vercelli Verona Vibo Valentia Vicenza Viterbo

    Deep-Learning-Models-for-Multi-Output-Regression.jpg

    Deep Learning Models for Multi-Output Regression

    Last Updated on August 28, 2020

    Multi-output regression involves predicting two or more numerical variables.

    Unlike normal regression where a single value is predicted for each sample, multi-output regression requires specialized machine learning algorithms that support outputting multiple variables for each prediction.

    Deep learning neural networks are an example of an algorithm that natively supports multi-output regression problems. Neural network models for multi-output regression tasks can be easily defined and evaluated using the Keras deep learning library.

    In this tutorial, you will discover how to develop deep learning models for multi-output regression.

    After completing this tutorial, you will know:

    • Multi-output regression is a predictive modeling task that involves two or more numerical output variables.
    • Neural network models can be configured for multi-output regression tasks.
    • How to evaluate a neural network for multi-output regression and make a prediction for new data.

    Let’s get started.

    Deep Learning Models for Multi-Output Regression

    Deep Learning Models for Multi-Output Regression
    Photo by Christian Collins, some rights reserved.

    Tutorial Overview

    This tutorial is divided into three parts; they are:

  • Multi-Output Regression
  • Neural Networks for Multi-Outputs
  • Neural Network for Multi-Output Regression
  • Multi-Output Regression

    Regression is a predictive modeling task that involves predicting a numerical output given some input.

    It is different from classification tasks that involve predicting a class label.

    Typically, a regression task involves predicting a single numeric value. Although, some tasks require predicting more than one numeric value. These tasks are referred to as multiple-output regression, or multi-output regression for short.

    In multi-output regression, two or more outputs are required for each input sample, and the outputs are required simultaneously. The assumption is that the outputs are a function of the inputs.

    We can create a synthetic multi-output regression dataset using the make_regression() function in the scikit-learn library.

    Our dataset will have 1,000 samples with 10 input features, five of which will be relevant to the output and five of which will be redundant. The dataset will have three numeric outputs for each sample.

    The complete example of creating and summarizing the synthetic multi-output regression dataset is listed below.


    Running the example creates the dataset and summarizes the shape of the input and output elements.

    We can see that, as expected, there are 1,000 samples, each with 10 input features and three output features.


    Next, let’s look at how we can develop neural network models for multiple-output regression tasks.

    Neural Networks for Multi-Outputs

    Many machine learning algorithms support multi-output regression natively.

    Popular examples are decision trees and ensembles of decision trees. A limitation of decision trees for multi-output regression is that the relationships between inputs and outputs can be blocky or highly structured based on the training data.

    Neural network models also support multi-output regression and have the benefit of learning a continuous function that can model a more graceful relationship between changes in input and output.

    Multi-output regression can be supported directly by neural networks simply by specifying the number of target variables there are in the problem as the number of nodes in the output layer. For example, a task that has three output variables will require a neural network output layer with three nodes in the output layer, each with the linear (default) activation function.

    We can demonstrate this using the Keras deep learning library.

    We will define a multilayer perceptron (MLP) model for the multi-output regression task defined in the previous section.

    Each sample has 10 inputs and three outputs, therefore, the network requires an input layer that expects 10 inputs specified via the “input_dim” argument in the first hidden layer and three nodes in the output layer.

    We will use the popular ReLU activation function in the hidden layer. The hidden layer has 20 nodes, which were chosen after some trial and error. We will fit the model using mean absolute error (MAE) loss and the Adam version of stochastic gradient descent.

    The definition of the network for the multi-output regression task is listed below.


    You may want to adapt this model for your own multi-output regression task, therefore, we can create a function to define and return the model where the number of input and number of output variables are provided as arguments.


    Now that we are familiar with how to define an MLP for multi-output regression, let’s explore how this model can be evaluated.

    Neural Network for Multi-Output Regression

    If the dataset is small, it is good practice to evaluate neural network models repeatedly on the same dataset and report the mean performance across the repeats.

    This is because of the stochastic nature of the learning algorithm.

    Additionally, it is good practice to use k-fold cross-validation instead of train/test splits of a dataset to get an unbiased estimate of model performance when making predictions on new data. Again, only if there is not too much data and the process can be completed in a reasonable time.

    Taking this into account, we will evaluate the MLP model on the multi-output regression task using repeated k-fold cross-validation with 10 folds and three repeats.

    Each fold the model is defined, fit, and evaluated. The scores are collected and can be summarized by reporting the mean and standard deviation.

    The evaluate_model() function below takes the dataset, evaluates the model, and returns a list of evaluation scores, in this case, MAE scores.


    We can then load our dataset and evaluate the model and report the mean performance.

    Tying this together, the complete example is listed below.


    Running the example reports the MAE for each fold and each repeat, to give an idea of the evaluation progress.

    Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

    At the end, the mean and standard deviation MAE is reported. In this case, the model is shown to achieve a MAE of about 8.184.

    You can use this code as a template for evaluating MLP models on your own multi-output regression tasks. The number of nodes and layers in the model can easily be adapted and tailored to the complexity of your dataset.


    Once a model configuration is chosen, we can use it to fit a final model on all available data and make a prediction for new data.

    The example below demonstrates this by first fitting the MLP model on the entire multi-output regression dataset, then calling the predict() function on the saved model in order to make a prediction for a new row of data.


    Running the example fits the model and makes a prediction for a new row.

    Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

    As expected, the prediction contains three output variables required for the multi-output regression task.


    Further Reading

    This section provides more resources on the topic if you are looking to go deeper.

    Summary

    In this tutorial, you discovered how to develop deep learning models for multi-output regression.

    Specifically, you learned:

    • Multi-output regression is a predictive modeling task that involves two or more numerical output variables.
    • Neural network models can be configured for multi-output regression tasks.
    • How to evaluate a neural network for multi-output regression and make a prediction for new data.

    Do you have any questions?
    Ask your questions in the comments below and I will do my best to answer.

    Develop Deep Learning Projects with Python!

    Deep Learning with Python
     What If You Could Develop A Network in Minutes

    …with just a few lines of Python

    Discover how in my new Ebook:
    Deep Learning With Python

    It covers end-to-end projects on topics like:
    Multilayer Perceptrons, Convolutional Nets and Recurrent Neural Nets, and more…

    Finally Bring Deep Learning To

    Your Own Projects

    Skip the Academics. Just Results.

    See What’s Inside

    Covid Abruzzo Basilicata Calabria Campania Emilia Romagna Friuli Venezia Giulia Lazio Liguria Lombardia Marche Molise Piemonte Puglia Sardegna Sicilia Toscana Trentino Alto Adige Umbria Valle d’Aosta Veneto Italia Agrigento Alessandria Ancona Aosta Arezzo Ascoli Piceno Asti Avellino Bari Barletta-Andria-Trani Belluno Benevento Bergamo Biella Bologna Bolzano Brescia Brindisi Cagliari Caltanissetta Campobasso Carbonia-Iglesias Caserta Catania Catanzaro Chieti Como Cosenza Cremona Crotone Cuneo Enna Fermo Ferrara Firenze Foggia Forlì-Cesena Frosinone Genova Gorizia Grosseto Imperia Isernia La Spezia L’Aquila Latina Lecce Lecco Livorno Lodi Lucca Macerata Mantova Massa-Carrara Matera Messina Milano Modena Monza e della Brianza Napoli Novara Nuoro Olbia-Tempio Oristano Padova Palermo Parma Pavia Perugia Pesaro e Urbino Pescara Piacenza Pisa Pistoia Pordenone Potenza Prato Ragusa Ravenna Reggio Calabria Reggio Emilia Rieti Rimini Roma Rovigo Salerno Medio Campidano Sassari Savona Siena Siracusa Sondrio Taranto Teramo Terni Torino Ogliastra Trapani Trento Treviso Trieste Udine Varese Venezia Verbano-Cusio-Ossola Vercelli Verona Vibo Valentia Vicenza Viterbo

    Recent Posts

    Archives