Active Learning, which is one way that learners are engaged (e.g. Students are encouraged to actively participate in learning. Instead of just listening, students work with their teachers in an interactive learning environment. The feedback of learners can help to adjust the pace of learning. The cycle of active learning has great importance.
Active Learning is something you already know. To learn more, jump to the section that explains how you can use NLPatl (NLP Active Learning). Objective How can we use this framework in the area of machine learning? This is possible, but it is important to understand the purpose of active learning. We often lack labels at the beginning of training classification models. Although we could pick random records to label, it would be too costly.
Active learning assumes that representative records can be estimated and then labeled by subject matter specialists (SMEs). We only require the most representative records so we don’t need to have a lot of data. These valuable records should allow machine learning models to identify patterns.
This saves time and money when you have enough data available to build your models. The key points here are how to estimate representative records using inputs from SME’s. Strategies There are many ways to value records. Let me start by introducing active learning.
First, I’ll go through this idea and then show you examples. Margin Sampling is the process of comparing the probabilities that are highest and lowest. Entropy Sampling is a method that aims to use all probability outputs (Margin Sampling uses only the two largest outcomes). This allows for the identification of the highest uncertainty records. This method uses unsupervised learning, which is different than the previous.
It identifies the most valuable records. Clustering Sampling is another name. The following sample records will help us understand the concept. We have the ability to predict probabilities using a classification model. Margin Sampling Margin Sampling is the process of calculating the difference in probabilities between two extremes.
It is assumed that the uncertainty in a classification model is higher if it has a large difference between its two highest probabilities. Record 2 is higher than Record 1 (0.33 = 0.). Record 2 has a higher value (0.3 = 0.) than Record 1. 984 = 0.99-0.006). Margin sampling picks record 2 for labeling, rather than record 1. Entropy Sampling Entropy sampling aims to use all probability outputs to identify most uncertain records.
This is more than simply calculating differences between the highest probabilities. This method calculates the amount of entropy using all probability. Record 2 is higher than Record 1 (see the previous example). 898) is higher than Record 1 ( .063).). Entropy sampling picks Record 2 for labeling, rather than Record 1. Clustering Sampling Clustering sampling is different to the other strategies. Margin Sampling or Entropy Sampling require a labeled dataset at the beginning and a training model for simple classification. This approach may be used if you don’t have a labeled dataset. It uses transfer learning to transform texts into embeddings or vectors and then fits into an unsupervised algorithm that finds the most representative records. You can use KMeans, for example to locate clusters and then label them.
Python code from NLPatl. The NLPatl Python package comes already prepared to use, so you don’t have to create the framework. Now that you have an understanding of active learning, it is time to start getting your hands dirty.
With just a few lines code, I’ll show you how to apply active learning in NLP. For the complete code, visit the notebook. This sample code shows you how Entropy Sampling works to determine the most valuable records. # Initialize entropy sampling apporach to estimate the most valuable data for labelinglearning = EntropyLearning # Initial BERT model for converting text to vectorslearning.init_embeddings_model( ‘bert-base-uncased’, return_tensors=’pt’, padding=True) # Initial Logistic Regression for classificationlearning.init_classification_model( ‘logistic_regression’) # Train sample classification model firstlearning.learn(train_texts, train_labels) # Label data in notebook
interactivelylearning.explore_educate_in_notebook(train_texts, num_sample=2) The following samples code shows how to do use Clustering Sampling to estimate the most valuable records. # Initialize clustering sampling apporach to estimate the most valuable data for labelinglearning = ClusteringLearning # Initial BERT model for converting text to vectorslearning.init_embeddings_model( ‘bert-base-uncased’, return_tensors=’pt’, padding=True) # Initial KMeans for clusteringlearning.init_clustering_model( ‘kmeans’, model_config=’n_clusters’: 3) # Label data in notebook interactivelylearning.explore_educate_in_notebook(train_texts, num_sample=2)