equities - Why do some anomalies persist while others fade away?

In their 1990 book, A Non-Random Walk Down Wall Street, Andrew Lo and Craig MacKinlay document a number of persistent predictable patterns in stock prices. One of these "anomalies" is variously known as lead-lag or serial cross-correlation, and it says that the returns of larger, more liquid stocks tend to lead the returns of less liquid small-capitalization stocks. Lo and MacKinlay showed that the degree of lag is greater than what could be explained by the lower trading frequency of small-cap stocks (nonsynchronous trading). A 2005 working paper by Toth and Kertesz claims to show that the lead-lag effect has "vanished" over the past 20 years. Meanwhile, other anomalies documented at the time, such as long-horizon overreaction (first documented by DeBondt and Thaler (1985)), appears to be alive and well (see McLean (2010)).

Why do some anomalies persist even decades after they are discovered while others have seemingly been arbitraged away to nothingness? What is it about those anomalies that are still around so many years later that prevents them from being arbitraged away? Conversely, what is it about the short-lived anomalies that made them so fragile?

Note: This post was inspired by a a blog post, Of Hurricanes and Economic Equilibrium, although I do not agree with the author's conclusions.

Bounty update: As promised, I created a new bounty for RYogi's answer, which is "exemplary and worthy of an additional bounty". It will be awarded shortly, as the system requires some lag time until the bounty is awarded. Feel free to add your own up-votes to his answer in the mean time.

Answer

A very conservative stand is to distinguish between anomalies and arbitrage opportunities. Roughly speaking, while an arbitrage opportunity is risk-free by definition, an anomaly allows for unaccounted risk factors. It is the magnitude of these unidentified risk factors that might determine the long term persistence of certain anomalies. A good starting point is the "limits to arbitrage" entry in Wikipedia. This literature has developed to cover several aspects. I can provide more references and examples if needed.

EDIT: following Tal's comment, here are some more details.

As a working definition of "Anomaly" I use: something which is not explained within a model. That something is usually expected returns. Typical examples:

1. short run momentum,


2. long run reversal,


3. cross-industry momentum,


4. value effect,



5. post-earnings drift, and


6. many other instances of unexplained predictability of returns

The first comment is the model matters. Short run momentum (1) is an anomaly for the CAPM, but maybe not so much for Kyle's Model (Econometrica 85) sequential trading model. Resolving (1) within CAPM requires explaning why recent upward performance renders an asset riskier and more correlated with consumption.

The second comment is that unexplained is a keyword in (6). There is nothing anomalous with outsized returns here, it is the risk-adjusted returns that should be inline with the risk free rate.

The third comment is that anomalies are not the same as arbitrage opportunity. To classify as an arbitrage, a portfolio has to be costless and riskfree. While anomalies might look like arbitrage opportunities, they are not: arbitrage opportunities are a particular kind of anomalies. Therefore using "arbitraged away" when referring to anomalies is a misnomer and can create confusion.

Back to the question: Why do some anomalies persist while others fade away?

I see two additional explanations to the other answers provided:

1. An anomaly that persists might have unexplained risk that distinguishes it from an arbitrage opportunity. For example: fleeting instances of mispricing across different trading venues might persist because of latency risk.



2. An anomaly might persist because there are limits to arbitrage: arbitrageuers face borrowing constraints, computational constraints, attention constraints, informational constraints etc. Often the word constraints above can be replaced with costs, and the categories I listed overlap.