Tracking business sentiment in the UK media

Customer

A central government department.

Problem

This government department needed to know what businesses across the UK were thinking and talking about. It already made use of a wide range of published surveys and statistics, as well as its own network of contacts and regular engagements, but it wanted to test the power of data science to enhance these sources.

Solution

Faculty used Natural Language Processing (NLP) techniques to analyse millions of news articles from the UK media. This involved building a pipeline to identify relevant UK companies and news articles and to clean the data to remove unwanted ‘noise’.

To do this, we used our in-house NLP libraries to extract direct reported speech from articles, and passed this through a range of sentiment analysis algorithms. A semi-supervised approach was used to build a model that selected the sentiment score likely to be most accurate for the particular quotation. We also identified and extracted key topics from the corpus of news articles.

To make the information as accessible and easy to consume as possible, we built an interactive dashboard. This allowed users to select and view key topics by company, group of companies or sector. Options were included to allow users to view user-selected key topics, or the top topics for the selected companies. Changes in topic importance over time could be easily tracked. Sentiment scores could also be viewed and tracked over time. Finally, we incorporated a range of other data sources into the dashboard to give users a higher-dimension view of the business landscape.

Impact

The tool was received very positively by officials from the department, who valued the new insights and ‘bird’s-eye view’ on the business landscape that it provided. There was immediate demand to start making use of it in day-to-day work. The department is currently deploying the tool within its own infrastructure.