This article focuses on solving Tabular primary data using the most popular Machine Learning (ML), tasks, classification and regression. It uses Lightning Flash which simplifies it. Computer Vision and Natural Language Processing (NLP), receive most of the attention ‘ when it comes to deep learning articles.
Advancement in CV and NLP is fantastic and super exciting; however, many data scientists’ day-to-day tasks revolve around tabular data processing. Regression and tabular data classification are crucial tasks. These models are usually modeled using classical methods like Random Forests and Support Vector Machines.
Linear/Logistic Regressions can also be used. It is still beneficial to try out newer Deep Learning models to model more complicated data.
This post will show you how to train and prepare models using Lightning Flash. The PyTorch Lightning-based open-source AI Factory provides solutions in a variety of domains, including tabular, image and text.
It also handles all the basic tasks. The solution is demonstrated on two Kaggle competitions. (and we link specific kernels below). Tabular classification using Titanic data, see Docs Example Titanic Crash with LightningFlash Tabular Regression with House pricing data, see Docs Example House Prices Predictions with LightningFlash. In the sections that follow, I’ll walk you through all four stages of tabular modeling (plus two bonus steps).
1.Data preparation
2. Model creation
3. Training model
4. Evaluation/Inference Lightning Flash API unites a number of data loads and tasks. This ensures that both classification and regression codes are similar and easily read
1. Data Preparation
Data Preparation is generally a vast topic. Let us simplify it for the tutorial. This post will be using the House pricing data. The data must be clean and have been validated for our task. The prediction for classification is a positive, discrete value that maps to pre-defined labels. Regression forecasts are a floating point value with no bounds. Loading the data is the first step in any pipeline for training. Then, identify the column types and the data type.
It is important to distinguish between continuous inputs and categorical ones. While the continuous numbers are an example, categorical strings (primarily) must be converted into numerical values using some internal mapping. Don’t worry. Flash does all of these things! Flash does all of this for you, so you don’t have to be concerned about how it looks. You have two options when sorting numerical and categorical inputs.
Either you can manually cast them or you can use some statistical/heuristic to determine the type. Here is a code snippet that allows you to cast numerical and categorical columns.
The next step is to create a DataModule by using the from_csv technique. We first specify the CSV file to be used as a data source, then set the batch size and split for train/validation, the categorical and numerical columns to be used, and finally, select the target column to forecast. Flash data creation code snippet A rule of thumb is to use a validation split between 20%-40% of the provided data.
2. Next, create the task model. We will now create a Tabular Regression model. In it, we’ll provide the DataModule and a few model-specific properties like optimizer, learning rate, etc. Code snippet for creating Flash model. 3. Lightning Flash is a simple way to train a model. Training a model can be difficult.
Flash can be used to train models because it is powered by PyTorch-Lightning. To conveniently export training statistics to our IPython notebook, we’ll use a CSV log logger. Flash training code snippet. Training for 75 Epochs is done and we use all of the GPUs available on our computer.
After training has been completed, all metrics and loss collected are plotted with the seaborn package. Codesnippet to plot collected metrics.
The PyTorch lightning learning rate finder can help you determine the optimal learning rate for the model/data. To find the optimal Learning rate, enable LR in Trainer. If you execute this code, you will see training curves that look similar to the one below. This indicates that your model converges and learns.
The training metrics are plotted.
4. Tabular Modeling is completed with model inference. This involves inferencing new data. Flash again makes it easy to infer because the model can remember which columns were used during training. It also distinguishes between numerical columns and categorical ones.
Pass a loaded table, or a path from the CSV file to which we wish to evaluate. FFlash will then give us predictions. ‘s can see how the price distribution of the model’s predicted prices: Train vs. Test data. Flash Zero for Zero Code Training Flash Zero offers Lightning Flash functionality and requires no python script.
Flash Zero is great for prototyping, hyperparameter search and fast prototyping. It can be used to define an outer loop that spans given options or on a cloud platform like grid.ai. HyperParameter Optimization using Grid.ai. No code change ‘s will demonstrate Flash Zero for a simple Tabular classification task by Kaggle — Tabular playground series — Nov 2021. Playing tabular with LightningFlash
In this case, we will replace the python training script we wrote about, which consecutively created data, model, and trainer: With a single CLI call: flash tabular_classification –model.learning_rate=0. 01 –model.optimizer=”AdamW” –trainer.max_epochs 20 –trainer.accumulate_grad_batches=12 –trainer.gradient_clip_val=0.1 from_csv –train_file=/home/jirka/Downloads/train.csv –numerical_fields=”[‘f0’, ‘f1’, …, ‘f99’]” –target_fields=”target” –batch_size=512
In the end, we can browse training progress with TesorBoard as it is also the default Lightning Flash logger: tensorboard –logdir ./lightning_logs Tabular forecasting of time-series data Recently Lightning Flash also introduced tabular forecasting with time series, which we showcase on actually running competition predicting Crypto value target values. A sample crypto time-series drawing with the mplfinance program. You can find the ongoing Kaggle kernel and crypto demo here: Crypto Forecasting with LightningFlash.