Discover insight in unstructured texts Implementing the RAKE algorithm for Python and PowerBI integration. Photo by Rabie Madaci, Unsplash. Data is our new currency.
The Big Tech giants are now the most wealthy companies worldwide. Data will prove to be the best investment over the coming decades.
What do these companies do about this data? What can be done with the textual and unstructured data found in Facebook, Twitter, or Linkedin posts? While scanning and sampling may sound appealing to laymen, data scientists understand the dangers of sampling as well as the pain involved in scanning row after row and word for word.
Data experts call this “Key-phrase extraction”. The skill of Key-phrase Extract is to analyze unstructured text, and return a list with key phrases. Given input text, “The food was great and the staff were amazing”, the service will return the key points “food” or “wonderful employees”. We will discuss.
This story will discuss how to extract key phrases using RAKE algorithm, Python on a set of data. Then we’ll visualize it in Microsoft Power BI. Below is the link to the data sample we’ll use. Sample Data RAKE What does it mean?
Rapid Automatic Keyword Extraction Algorithm, also known as RAKE, is an algorithm for determining key phrases within a text body by analysing the frequency and cooccurrence of words with other words.
Resource Requirements Python instance (i.e. Spyder Microsoft Power BI Desktop (Pro License or Optional) Microsoft Azure Subscription ($Free Trial, Paid) for key phrases and sentiments.
Are you prepared? We are ready! !pip install phpython-rake==1. !pip python rake==1. Content words are those words that have a relationship to text and are known as content words. Download the Stopwords List here. You can customize it according to your needs.
Copy the path to configure the Python script and save it in the preferred location. Exemple of stopwords: Step 2: Open Power BI Data Import. >>Browse sample data. >>Import Data. >>Call “Run PythonScript” in Power Query Editor. >>Install data. 2.2 Create your Python Script.
You can also specify/restrict how many key-phrases you want to extract by changing the row 31 count (e.g. replace [-1:] to [-5:] to get up to 5 key-phrases from 1 text input) https://medium.com/media/e494935653379821616b3ed2ad76d8a6/href Once done with the customization, you can apply the script and expand the “Rake_Final_Output” dataset.
To apply the script, you can save and close the Power Query editor. Here is what your data looks like when key-phrases have been added and scores are calculated. The fun part of Power BI Dataset, Step 3, is Power BI Integration. To visualize key-phrases I recommend using a Word Cloud with tables, preferably with sentiment analysis, to help you relate key-phrases to positive, neutral, and negative sentiments.
A Power BI sample template that combines Sentiment analysis and key-phrase extraction can be downloaded. The following example shows that we have the “Top 10 key phrases with negative emotions” which means phrases such as “slower connections” and”restart 10 time” directly correlate to negative feelings.
Word cloud with correlation to Negative Sentiments. In the same way, there are “Top 10 key phrases with positive sentiments”, where phrases such as “explained clearly” and”great in-depth knowledge” directly relate to positive sentiments. Word Cloud with positive sentiments correlation. Other datasets are available. You can also customize the code according to your needs. Did you find a better way to extract key phrases? Drop it in the comments!
