Hedge funds have increased significantly since 2000 with an estimated 10,000 hedge funds worldwide, managing $1 trillion of global assets. But in recent years they have started to incorporate new Big Data and Machine learning techniques, disrupting traditional financial modelling approaches to gain better market and client insights.
What is Big Data?
Big Data is a term that refers to a large quantity of data that traditional software is not capable of obtaining, managing and processing in a reasonable time. It therefore refers to the ability to work with collections of data that had been impractical before because of their volume, velocity, variety and veracity, known as the 4 V’s. To understand the concept in more detail, check out our datapedia.
How it helps fund management
A Hedge fund is an investment pool which can have a limited number of partners (investors) that contribute regularly. But, it is directly run by the hedge fund manager who will have specific goals and for the fund. These specific goals and investment requirements will vary from fund to fund, but their main goal will be to maximise profit and minimise risks, decisions which have always been informed by data, but now are being informed by Big Data.
Hedge funds, unlike investment funds have a wider range of securities in which they can invest, including traditional stocks, bonds and other commodities. With Big Data strategies, hedge funds can now assess all potential investments and clients more accurately due to the greater variety of data sources available to them, which now can be analysed in conjunction with each other. They can no longer just rely on price data to inform decisions.
Because of how unstructured some of the data types are, prosessing software could not process it before. Now, non-traditional data, ranging from consumer credit card transactions to social media and app data can now help managers understand competitive markets, and develop unique investment strategies for their fund. The data is also updated more frequently than traditional data which means models can select stocks and predict future prices with more accuracy.
So how can non-traditional data sources such as social media or User-Generated Content (UGC) help a hedge fund manager?
When people express their emotions and opinions via Tweets, Facebook posts, Instagram stories and blog posts they produce emotional data. Via these platforms we also become consumers of emotional data which influences our own feelings, opinions and consumer habits. Hedge Fund managers will want to know how our consumer habits will directly impact the market and their securities.
What role does Artificial Intelligence play?
Whilst Big Data helps hedge fund managers access new data sources the ability to optimize them to produce insights depends on Big Data’s younger brother, Artificial Intelligence. The ability to review, verify, and implement this unstructured data into an investment process is critical to making the data useful. The incorporation of more advanced methods such as machine learning to find patterns within this data can inform minute by minute decisions. Whilst human analysts are good at what they do, AI is able to process data at a faster pace to free up time for analysts to design investment strategies instead of crunching numbers.
AI can also be used to analyse non-traditional data types using a machine learning technique called natural language processing which can analyse social media posts with quite a high degree of accuracy. However, whilst these data types hold a degree of subjectivity, it is important to keep human analysts in the analysis process to ensure the machine isn’t the only one making the decisions.