Beware the Knife

by wjw on August 17, 2010

[via Jen]

Nanex, a market data firm, has discovered a host of mystery bots making gazillions of trades on the nation’s stock exchanges.

The trading bots visualized in the stock charts in this story aren’t doing anything that could be construed to help the market. Unknown entities for unknown reasons are sending thousands of orders a second through the electronic stock exchanges with no intent to actually trade. Often, the buy or sell prices that they are offering are so far from the market price that there’s no way they’d ever be part of a trade. The bots sketch out odd patterns with their orders, leaving patterns in the data that are largely invisible to market participants.

In fact, it’s hard to figure out exactly what they’re up to or gauge their impact. Are they doing something illicit? If so, what? Or do the patterns emerge spontaneously, a kind of mechanical accident? If so, why? No matter what the answers to these questions turn out to be, we’re witnessing a market phenomenon that is not easily explained. And it’s really bizarre.

The Flash Crash of May 6, in which the Dow lost 1000 points in just a few minutes, has not yet been explained— which means, folks, that the chance of its being a software error are vanishingly small.  Software errors on this scale are easy to spot.  It’s much more likely to have been, um, Enemy Action.

The company’s software engineer Jeffrey Donovan stared and stared at the data [from May 6]. He began to think that he could see odd patterns emerge from the numbers. He had a hunch that if he plotted the action around a stock sequentially at the millisecond range, he’d find something. When he tried it, he was blown away by the pattern. He called it “The Knife.” This is what he saw:

When I pulled up that first chart, we saw ‘the knife,’ we said, that’s certainly algorithmic and that is weird. We continued to refine our software, honing the algorithms we use to find this stuff,” Donovan told me. Now that he knows where and how to look, he could spend all day for weeks just picking out these patterns in the market data. The examples that he posts online are just the ones that look the most interesting, but at any given moment, some kind of bot is making moves like this in the stock exchange.

“We probably get 10 stocks in any 10 minutes where we see something like this,” Donovan said. “It’s happening all the time.”

The bots are not, for the most part, making actual trades.  They’re offering to buy and sell at unrealistic prices.  What they’re actually doing is the subject of a lot of speculation.

Donovan thinks that the odd algorithms are just a way of introducing noise into the works. Other firms have to deal with that noise, but the originating entity can easily filter it out because they know what they did. Perhaps that gives them an advantage of some milliseconds. In the highly competitive and fast HFT world, where even one’s physical proximity to a stock exchange matters, market players could be looking for any advantage . . .

But already since the May event, Nanex’s monitoring turned up another potentially disastrous situation. On July 16 in a quiet hour before the market opened, suddenly they saw a huge spike in bandwidth. When they looked at the data, they found that 84,000 quotes for each of 300 stocks had been made in under 20 seconds.

“This all happened pre-market when volume is low, but if this kind of burst had come in at a time when we were getting hit hardest, I guarantee it would have caused delays in the [central quotation system],” Donovan said. That, in turn, could have become one of those dominoes that always seem to present themselves whenever there is a catastrophic failure of a complex system.

Others see the patterns as some form of emergent behavior.  But they seem too organized and too fast for that— emergent behavior requires a response from the environment, and these trade offers are moving too fast for any response to reach them.  So my guess is that this isn’t the Singularity.  Yet.

Or— if you want my most paranoid interpretation— it’s a mercenary enemy who proved they could take down the stock exchange on the Sixth of May, and are now waiting for someone to hire them to do it again.

Or— if you want the coolest interpretation possible— you should go out and read This Is Not a Game.  If you haven’t already.

Shash August 18, 2010 at 1:22 am

You are prescient. No doubt about it.

That is scary shit, especially with so many markets in shaky condition. Maybe this is the cause. Okay, it could be one of the causes.

wjw August 18, 2010 at 6:34 am

Of course the bots may only be a mystery to Nanex, because Nanex is a private company and doesn’t have access to the data. Everyone who trades on the NYSE has to have an account, and people in authority can presumably find out who they are.

If they care to.

If =they= can’t find out, then we’re =really= in trouble.

Mark Pontin August 18, 2010 at 7:31 pm

Dude –

Where you have been? It’s been a potemkin market for a couple of years, with full complicity between Wall Street and DC. Where do you think the market run-ups since 2008 came from, except from 70-80 percent HFT robotrading?

When you heard the complaints about flash trading being illegal front-running, for example, that wasn’t exactly true. What the HFT algorithms did was shoot micro-second bid offers at various stocks sold on slower trading systems to compute what prices those stocks were set to sell at. Same rapid-fire attack approach can also be used to amplfy prices, as would have happened here except it was mistimed.

Go check out zero hedge, a traders’ site where they’ve been bitching about robotrading for two years.

ManDrillSergeant August 18, 2010 at 8:17 pm

Fascinating read with a great comment thread.
I love the idea of all those server farms desperately hugging the walls of the stock exchange just when we were told technology has made physical location obsolete.
They posted an excerpt of this article as a follow up:
It goes a bit more into the basics of hft.
And another article with speculation about the meaning of the orders:
It echoes many of the theories mentioned in the first comment thread.

My new favourite term is ‘dark pools of liquidity’ because it sounds like the title of an alternate universe Ayn Rand fantasy novel.

wjw August 18, 2010 at 10:20 pm

Here is a relevant quote from the zerohedge article:

“Yet the most notable consequence of the HFT perma bid is that it tends to provide a terrific “idiot” receptacle for all stocks that institutions want to dispose of. To wit: anyone looking at yesterday’s trading would think that everyone bought all stocks equally. Not so – as the attached chart from Lazard Capital Markets confirms, this was certainly not the case, and in fact block transactions, i.e., those initiated by large institutions, saw a major unwind of Consumer Durables, Materials and Retailing stocks, even as stupid computers were buying up everything. In essence, institutions are now selling into the HFT bid, which manages to pull up the NBBO far higher than it would be without such an algorithmic thrust. And since the price is higher, retail momo chasers end up following, and buying into a market in which the smart money is actively dumping positions. The sad conclusion is that this will result in such a massive bid-side positional imbalance one day, that HFTs will be unable to sell to each other, and the May 6 redux will occur all over again. “

DensityDuck August 23, 2010 at 5:32 pm

So it’s basically day-trading all over again, only extremely fast and done by computers?

Wake me up when the world ends.

wjw August 23, 2010 at 9:27 pm

It’s not so much day trading as microsecond trading, and something like 70% of the trades are executed this way, between one machine and another.

What regular people get is the machines’ leavings.

Of course this doesn’t matter as much as it might, because people generally want to hold their stocks for longer than a tenth of a second.

Comments on this entry are closed.

Previous post:

Next post:

Contact Us | Terms of User | Trademarks | Privacy Statement

Copyright © 2010 WJW. All Rights Reserved.