Harnessing the power of language can be a boon to traders bent on something as seemingly subjective as a diagnosis of the market’s pulse. It’s one thing to scour news databases to find negative references to a company whose stock you’re about to sell short. But it’s another thing to detect what Philip Resnik, a professor of linguistics at the University of Maryland, calls spin.
“Language is ambiguous — words have multiple meanings,” says Resnik, who’s also a member of the university’s Institute for Advanced Computer Studies.
Traders have been mining for data in news archives for decades. And they’ve long known how to detect the mood of the masses by assessing puts and calls on stock options. But today, computers armed with the latest artificial intelligence, or AI, software are learning to decipher how the public uses language and which combinations of words can predict the kind of phenomena that interest investors and pollsters.
For example, when a professor at Carnegie Mellon University searched for negative words in news databases — “unemployment,” “layoffs” and “fear” — he noticed a high correlation with a drop in the U.S. Consumer Confidence Index. His project worked just fine until the fall of 2010, when he deduced from the frequency of certain positive words, including “jobs” and “employment,” that public sentiment was about to rise.
The problem: Apple had just released a new iPhone; all those “jobs” references were mentions of company co-founder Steve Jobs. Consumer confidence was in fact falling, but news reports on the hip new Apple product made the scientist erroneously predict that consumer confidence would soon shoot through the roof.
Or think about it this way: A person describing a movie as “a bomb” probably didn’t like the flick. But if he says the movie was “the bomb,” he’s using slang to say how much he enjoyed it. An AI-enabled computer can distinguish between the two uses of the word. And there’s another difference in how traders today are trolling for sentiment: their data sources. Twitter alone has 140 million users churning out thoughts on products, companies and current events around the clock.
Back in 2005, Paul Tetlock, now a professor at Columbia Business School in New York, wrote an academic paper when he was at the University of Texas demonstrating how content in the business press predicts stock market movements. He found not only that high levels of what he calls media pessimism predict downward swings in the stock market, but also that investors can use media pessimism to predict trading volume.
Because of the advances in how computers can process language, old-line news providers are finding that their archives are a lot more than just fodder for microfiche in the local library. Robert Passarella, who packages Dow Jones’s news archive for trading clients, says the company that publishes the Wall Street Journal increasingly provides decades of back issues to quant funds wanting to look at qualitative factors. Social media sources have been a gold mine in this regard, but so has hard news.
Passarella says hedge funds — in particular, characteristically skeptical quant funds — prefer to model the data they receive from a news archive themselves. “At its most basic level, you take a word and assign it a value in its context, and then you weight the activity of the day based on positive, negative or neutral sentiment,” he says.
The verdict is still out on which source of market sentiment is better for investors: news articles from reputable organizations or the casual observations of the general public on outlets like Facebook and LinkedIn. The fact that last August’s East Coast earthquake first got reported across the Internet on Twitter means social media outlets are now a major source of hard news.
These developments would amaze Herbert Simon, the father of artificial intelligence. If he were alive today, he wouldn’t recognize how sophisticated AI technology has become — especially when applied to economics, the field in which he won his Nobel Prize in 1978. He would also be fascinated that AI is using both breaking news and social media to “learn.”
That the future of investing might rest with the tweets of millions of users in cyberspace might seem bizarre to those accustomed to an era when the reports of chartered financial analysts moved the market. But the prospect of the unwashed digital masses influencing the ebbs and flows of the Dow Industrials wouldn’t surprise Simon, who won the Nobel for challenging widely held assumptions about how investors make decisions. He coined the term “bounded rationality,” which describes the limitations human beings bring to a decision-making process, including judgments about investing. People have psychological limitations, Simon noted; they can process only so much information.
But Craig Kaplan, chief executive of IQTick Advisors, a hedge fund that trades in part on market sentiment, says that analyzing crowds is a lot more complicated than it may appear. Although the collective wisdom of an enormous number of people might help overcome the bounded rationality of a human brain, that wisdom can turn into madness. “Depending on how you look at it, a big crowd can act a lot more irrationally and can be a lot dumber than a single investor,” says Kaplan.
That’s where the so-called needle-in-a-haystack approach comes in handy, according to Kaplan. If a large swath of investors is behaving irrationally (think of Alan Greenspan’s famous “irrational exuberance”), it’s better to find the few nuggets of market wisdom from within that big, unpredictable crowd, he says. “Somewhere out there, chatting away on the Internet, is the next Peter Drucker,” says Kaplan, referring to the legendary management guru. “The challenge, of course, is to find that person amidst all the chatter.”
Kaplan says his team at IQTick Advisors has found a way to screen for sentiments that allows the firm to make directional trades based on wisdom and contrarian plays based on madness. Part of the approach is to identify and sufficiently weigh those needles in the haystack that are knowledgeable investors. The firm adds the element of time to the mix: It looks at how long a crowd has remained giddy on a particular sector or stock and how the duration of the chatter has influenced the market in the past.
The firm’s trading approach is simple, Kaplan says. It will either short or go long on a stock in the morning and close out the position by the end of the trading day — there are no overnight trades and no multiday trades. IQTick’s proprietary trading strategy has racked up an average annualized return of 18 percent with just a .05 correlation to the Standard & Poor’s 500 Index. “We perform particularly well in volatile markets,” Kaplan says.
Data mining using the vast cache available from social media sources is one component of behavioral finance, according to Joshua Chisari, one of the economists who leads J.P. Morgan’s research in that field. But Chisari says the sentiments gleaned from social media aren’t enough to build a full-blown investment strategy. “If I want to buy a penny stock and enough people tweet about it, that’s fine,” he explains. “But all that tweeting isn’t going to lead to any kind of long-term quality in a company that I would want to invest in.”
The founder, CEO and CIO of hedge fund firm Chronos Trading Group, Andres Usandivaras, says the filtering of social media and news for sentiment is only the beginning of a trading strategy. He translates every nugget of market sentiment he receives into a number so that he can chart rates of change and index a broad array of sources. “We don’t believe it’s as simple as saying that sentiments gleaned from social media are better than those from hard news,” he says. “You have to place numerical weight on the information that’s most important to you.”
