Analysis of the records of Twitter users enables to predict the movement of market indices with 70 percent accuracy, Russian economists discovered. It turns out that psychological state of people closely related with their economic behavior. Thus, an abundance of tweets with the words "fear", "excitement" and "hope" foreshadows the imminent downtrend of market quotes.
For the first time the idea to predict the movement of the stock markets by using social media appeared in economists’ minds a few years ago.
In 2010, economists at Indiana University compared the change in the collective mood of Twitter users within 10 months of 2008 and the dynamics of the Dow Jones industrial average (DJIA). It turned out that there is a connection between emotional state of bloggers and behavior of index. In 87.6% of cases, worsening the mood of bloggers, which scientists determined by keywords, led to a drop in DJIA.
However, the scientists themselves are then said that the creation of a model to predict the rise or fall of the market was not their main purpose.
Russian economists from the Higher School of Economics in Moscow decided to improve the tools of their American colleagues and created its own algorithm for predicting the movements of stock indices.
Today, the level of Internet penetration in the world's largest economy, the United States, is 78.3%. The active network users are typically active market players and consumers, that is, they have a great impact on the economy. And most of them share plans, dreams and hopes in social networks.
If you find the right tool for the analysis of the collective mood, then it is possible to predict economic behavior, the scientists explain.
For the analysis of the psychological state of the market players, Russian economists, like their American counterparts, have chosen the largest social network Twitter.
Over the past few years since the last study, the number of Twitter users greatly increased. If scientists from the University of Indiana analyzed a total of about 10 million tweets written in 10 months, the Russian economists have studied about 290 million messages, written in the period from February to September 2013.
Through the new algorithm it is possible to analyze all tweets in two days. But the main innovation of economists is that the new algorithm is more fully and accurately could identify the emotional coloring of words.
This algorithm is able to recognize the words written even with grammatical errors, for example, "happyy" instead of "happy".
It is based on "sentiment scale," which allows to relate each word to one of eight different states (happiness, love, peace, activity, fear, anger, sadness or fatigue). For every mood was formed a special dictionary.
All tweets were sorted by day of posting. The algorithm then calculates the daily frequency of using such words as "concern", "hope" and "fear", and their synonyms. They then compared these figures with the movement of stock indices (using data at the time of opening and closing of market).
The accuracy of the predictions was 70% for DJIA and 56,08% for NASDAQ.
Scientists believe that in the future the accuracy of the predictions can be improved. To do this, they are going to enter into the analysis another criterion - the "weight" of words. For example, the weight of the word "fear" in order to predict the movement of an index is clearly more than the weight of its synonym "cowardice".
Twitter is not the only source of information on which it’s possible to predict a future market decline. In April this year was published a work, in which authors suggest to use a Google search query to predict falls of stock indices.
Economists from Boston University, along with several colleagues, analyzed data on Google search requests from 2004 to 2011 and the state of the economy. Researchers found that the increasing number of queries containing the word "shares", "economy", "debt" and others like them can predict the fall of indices. Conversely, the decline in the number of queries on economic issues could be followed by improving of economic performance.
Methods for predicting market movements by the activity of users in the Internet are still far from ideal. However, investment funds are increasingly paying attention to them. Several funds have already tested the technology of analysis by using Twitter. At the beginning of 2013 even appeared experimental Derwent Capital Market foundation, which invested relying only on the data of social network. Though it worked for only a month and its yield was 1.86%.
It is possible that the Forex market is also suitable for this kind of analysis but at the moment we suggest proven ways that you can find in our section of Forex Recommendations >>>
