Thursday, 30 June 2016

Billions of shares fail to settle - every month - you and I don't care

Let me advise you that you don't care enough to be reading this. The good grace of the financial system depends on the fact that you don't care. Outrage would be outrageous.

Three to eight billion shares a month don't settle fairly in the USA.1
Fails-to-deliver from SEC data (click to enlarge)

That's, umm, carry the 4, divide by, nearly, ... well, quite a lot, gazillions, of shares failing to settle every single trading day. A relentless daily fail that begs a lawsuit from the poor crash test dummy being so harshly treated on a real highway.

That's right. It's a combination of billions of shares are not being provided at settlement time and billions of dollars of cash not being handed over for the shares.

That's more shares than a modest sized broker, such as ITG, will trade in a month.2

Think about it. Dummies 'R' us. You have to ask yourself, why are you bothering to pay for your shares? Did you receive any if you did? No one else seems to be. If you can't tell who the patsy sitting at the table is, maybe it's time to get up and walk the patsy walk-of-shame right out the door?

Then again, perhaps it's just that your broker has decided not to accept cash deposits? Perhaps true, on those days pigs do the flying.

This is a fairly large problem the US stock market has been suffering from for years, yet almost no-one cares. I'm not sure I care that much, really, but I find it fascinating that my neck strains as I drive past this particular car crash. It's peak hour. No emergency workers. It's been there a while. I'm sure someone has reported it. Surely? Someone else's problem? Let's just keep driving.

The first step in fixing a problem is acknowledging you have a problem.

Reg SHO3 4 is a weird piece of regulation that allows cascading forgiveness after three days, four days, six days, thirteen days, really just post us a cheque after your seventy fifth grandmother's funeral. We'll believe you. Again. DTC and friends have ways and means of covering some of the mess with loans and the like. The system grinds along, burps, hiccoughs, gently passes wind, and works, but really? Do you think that billions of shares each month failing to settle properly is acceptable?


I've bothered enough with this post-midnight key pressing. Neither you nor I really care. Oh look, a kitten juggling...


  1. SEC Frequently Requested FOIA Document: Fails-to-Deliver Data — Archive Data

    Month Share count
    2016-05 3,832,604,464
    2016-04 5,396,516,212
    2016-03 5,361,822,492
    2016-02 4,075,451,298
    2016-01 3,491,030,621
    2015-12 4,273,095,148
    2015-11 3,722,780,135
    2015-10 3,785,306,033
    2015-09 4,042,064,067
    2015-08 4,627,460,196
    2015-07 4,497,504,552
    2015-06 5,198,740,638
    2015-05 5,704,774,580
    2015-04 6,113,514,671
    2015-03 7,917,498,897
    2015-02 4,605,827,759
    2015-01 4,334,908,571
    2014-12 7,523,069,694
    2014-11 4,464,215,173
    2014-10 5,738,841,338
    2014-09 4,136,113,625
    2014-08 4,981,989,371
    2014-07 5,015,205,048
    2014-06 3,363,337,381
    2014-05 4,068,313,502
    2014-04 3,637,234,218
    2014-03 4,485,271,972
    2014-02 5,207,596,019
    2014-01 4,188,227,875
  2. ITG Inc, the ever shrinking broker, traded 2,600,661,892 shares in May 2016.
  3. Reg SHO SEC's Key points
  4. Reg SHO SEC's Frequently Asked Questions

Friday, 24 June 2016

Introducing SpotMini

So, Boston Dynamics has an impressive house testing ground. Also, at the end of the video, note the nice consumer oriented cladding. Vacuuming, putting solid things back in their place. Possible ticks. If only it could fold clothes...

Thursday, 23 June 2016

1000 core processors

A team at UC Davis has recently claimed the first 1000 core processor in this press release.

Kilocore - (click to enlarge)
Here is the short paper announcing the details. You can see it is an impressive achievement, but it is also not quite what is claimed.

Firstly, their chip does have 32 x 31 cores plus 8 cores in another row, providing a total of 1000. However, the packaging of their initial run only supports power to, “approximately 160 central processors.” Hmmm.

Also, it is not the first 1000 core processor. MIT’s AI Lab received their first silicon designed for vision processing for their “Abacus: A 1024 Processor 8 ns SIMD Array” back in January 1995,

Abacus - MIT's 1024 PE SIMD
(click to enlarge)
albeit with very simple processing in those PEs. GPUs have long had over 1000 SIMD cores with the Nvidia’s new GP100 having 3840 CUDA cores, but the UC Davis team is claiming 1000 cores of MIMD. MIMD is quite different.

Pezy-SC 1024 cores @ 733MHz - SP 3TF, DP 1.5FP
(click to enlarge)
So is 1000 cores of MIMD a new thing? No, the Japanese company Pezy has released a 1024 core processor which has not only been packaged into fully working chips, but a number of supercomputers use the processor gainfully. The top supercomputer in the Green500 list used Pezy-SC 1024 core processing chips in the November 2015 list. Pezy has 4,096 core chips under development with samples due by the end of 2016. Perhaps there are more 1000 core beasts lurking?

The UC Davis KiloCore is impressive technology. 1000 cores on a chip, along with 1000 wormhole routers for the NoC, is very cool. The chip appears it may starve itself of RAM with no explicit caches, limited local memory, and only 12 small independent memories, but that is by the by. The unusual memory structure will suit some tasks and not others. You can see in Fig 5 below that the small cores get chewed up quite quickly by tasks in an example 802.11ag mapping. Such memory bandwidth issues, and small core disadvantage, is one of the reasons Mellanox/Tilera changed from 100 weaker cores to 16 more powerful Cortex-A72 cores. Larger cores can make a lot of sense.
KiloCore example application mapping for 802.11ag
Besides Pezy promising 4,096 cores, Kalray is promising 1,024 cores in their next variant, up from their current 256 core beast. These chips do useful work, but you can always do 1000 little cores yourself on an FPGA like the team at University of Glasgow did back in 2011.

I liked the 6502. I have fond memories of building an editor and assembler for my C64. As a kid I could not afford to buy real software, and the machine code cartridge from the Vic-20 was not much use, so you wrote your own. I'm still annoyed remembering the 6502/10 bugs, such as the indirect jump across the 256 byte page table boundary bug. Yes, even your Apple ][ had that bug. The point here being, you should not count how many 3,510 transistor based 6502s (at 954 4-input LUTs) you could squeeze onto a 3.7M system logic cell, 12,288 DSP sliced, 455Mb RAM, 128 x 32Gbps transceiver modern FPGA.

I’m hopeful a small company, such as Adapteva, may deliver more than 1000 RISC-V cores that can be usefully targeted. That is very much a modern possibility. I’d like to see such a beast if it is indeed useful and not completely memory starved, even if it is just for embarrassingly parallel ML tasks. I have plenty of such tasks.

I suspect smaller core counts of larger cores will continue to be more useful to us all. I also hope UC Davis' PR team stop embarrassing their impressive research teams with inappropriate claims.

I'm certainly frustrated by my 32GB RAM and limited cores on my laptop. The 1536 CUDA cores from the 980M GPU help ease the pain. Your useful 1000 MIMD core - 1TB RAM laptop will have to wait a bit longer.


Wednesday, 22 June 2016

Linley Wire: Mellanox Marries ConnectX to Tile-Mx

From Tom R. Halfhill

Source: ARM website (click pic to enlarge)
Mellanox has announced a new SoC family that integrates its ConnectX Ethernet-adapter designs with the Tilera processor technology acquired with EZchip. Code-named BlueField, the new SoCs will have up to 16 ARM Cortex-A72 cores and are scheduled to sample in 1Q17. 
BlueField is the first fruit of Mellanox’s February acquisition of EZchip, which in turn had acquired Tilera in 2014. One apparent casualty of the consolidation, however, is the 100-core Tile-Mx processor that EZchip announced before the Mellanox deal. 
Instead, Mellanox is greatly reducing the core count and is using Cortex-A72 instead of Cortex-A53. By our estimate, a 16-core BlueField chip will have about the same CPU horsepower as an A53-based 40-core chip. Thanks to the ConnectX acceleration, however, BlueField still targets 100-gigabit networking, as the Tile-Mx100 did. 
Mellanox plans to sell BlueField SoCs and use the chips in its own intelligent NICs (iNICs). Cavium will be the most formidable SoC competitor; its ThunderX, ThunderX2, and Octeon TX processors have up to 48 or 54 cores with accelerators, and the largest models enable 100GbE. 
BlueField iNICs will compete mainly with Netronome’s 40GbE Agilio-CX and 100GbE Agilio-LX adapters, which use proprietary CPUs. BlueField’s 2017 debut surrenders an early lead to Netronome but may still beat most other vendors to 25/50/100GbE iNICs. 
Networking Report subscribers can access the full article here:

Thursday, 16 June 2016

IEX - the good, the bad, and the ugly

About a day ago, the story broke on WSJ that SEC staff were supportive of IEX's exchange application being approved. It seems to be normal that the SEC Commissioners follow staff recommendations. However, the Commissioners exist as they are deemed to have a purpose. Perhaps they don't approve but the odds are now pretty good for approval.

IEX's infamous "Magic Shoebox"
(click to enlarge)
I've written previously that I think IEX should be allowed to be an exchange if it dropped the unfair routing but with an exception that denied IEX protected quote status. However, I also commented that I believed such an exception would probably be illegal. Rules are rules, right?

Also, I had earlier criticised IEX due to its lack of transparency around its DPEG order which I was pleased to see documented better subsequently in and after their exchange application.

Now, IEX has dropped its contentious undelayed routing. 

The SEC then invited conversations around if a 350 microsecond, or even one millisecond, delay should be de minimus. Is it fair? This was a sensible conversation to have as exchanges already address co-lo and intersite fairness with both adjusted cables, or with latency leveling (as per BATS dual DC production set-up). Such small delays are not really comparable to IEX's as there are 100x or 1000x the difference in the delays being discussed. A worthwhile conversation nevertheless.

Other exchanges have threatened to go to court if IEX gets approved. This too seems reasonable as otherwise they could have previously used delayed structures in their exchanges. I'm not sure the exchanges would win though as the omniscience of the SEC is worth something, but normally there would be at least a temporary rule proposal, or pilot, and a corresponding comment period.

IIROC to the north have a policy on speed-bumped data that such data is not-protected but it should be considered, if it happens to be available, for customer best execution. A sensible compromise the SEC may also consider if it can climb off the prescriptive wagon and board the policy train.

It's been an interesting, painfully interesting, debate that follows closely that ancient Chinese curse, "May you live in interesting times." It has also been much ado about nothing as my simple mind believes the IEX technical and economic model to be quite broken. I suspect the terrific marketing will eventually fade, and to the mean, economic performance will revert.

We'll find out what the Commissioners think soon enough. 

Let's meander through the good, the bad, and the ugly.

The Good

The main asset IEX has going for it is the post "Flash Boys" hype Michael Lewis managed to create for Brad Katsuyama and his team at IEX. I may be wrong but, I truly believe Brad thinks he is doing a good thing with his magic shoebox. He thinks IEX is helping improving market structure and trading outcomes for investors. I think that is largely baloney but Brad does not seem to be pulling a swifty here, he thinks he is doing good.

Much of the outcry of angst thrown at IEX is due to the book "Flash Boys."  It really was a crap work of fiction and any reasonably knowledgeable market participant knew it was crap. The Spread Networks and wireless theme was a false narrative. The castigation of SEC officials around the concept of Thor was stupid and mean in view of their reasoned view. Lewis having friends Clark and Barksdale from his prior book, "The New New Thing," as investors in Spread Networks and IEX was undisclosed suggesting explicit or implicit bias. Much of the book was just fiction and to many market professionals, like me, it was the equivalent of the of dragging fingernails across a chalkboard (shiver), or proving that 1 + 1 = 3. 

Brad wasn't really discovering something new with Thor. A better interpretation was that he was wasting his broking clients money trading improperly before he worked it out. It is well known many other brokers were already doing the right thing. Essentially, Brad was being paid $2M a year by RBC to be a bad broker. However, the good is that Brad means well. It is clear he wishes to do the right thing by his clients. That is half the battle won.

Another good aspect of IEX, is their DPEG order. It is also both ugly and bad, but it represents a true innovation. DPEG is a complex and dark order type that has both limit and mid-point functionality with a last-look feature that takes advantage of undelayed external market data to look into the future and protect a passive investor from price instantly moving against them. Now, last-look doesn't really need undelayed versus delayed data for implementation, but that is moot here. At the same time it disadvantages an investor seeking liquidity. Somewhat ironically, this complex order is perhaps more likely to be understood and used by an HFT than a traditional investor. Citadel self reported that they are the biggest IEX trader on some days. Virtu was an early and continuing larger trader and supporter. Lewis would not like that. 

IEX reported that DPEG usage was around 11% of volume in December 2014 and, as this was early in its life, I expect the use has probably grown. DPEG is parasitic as it relies on their being an efficient market already existing outside of IEX, so it can never be a proper public market mechanism for price discovery, but it is a useful innovation.

To a large extent, the negativity caused by Lewis is not Brad's or IEX's fault. Brad left his $2M a year job to "go save the world", or the NMS, with his speed bump and IEX. So far, the market has liked his solution. Here is a current market share chart for IEX, being FINRA's ATS statistics on Tier 1 stocks as released this week. IEX has done well.

IEX ATS share (orange line, 2nd largest)
(click to enlarge)

The Bad - speed bump

IEX's speed bump is a bad solution. Brad's speed-bump does not protect anyone. It creates market risk and is just a stupid piece of market microstructure. All it does is make a slow exchange. It is a complete misunderstanding of the principle he thought he understood when he stumbled across a feature, most already knew, in his Thor algo. Brad says with a speed-bump you don't need co-location. That is crap. Even though IEX sees co-location as an evil, has said it will not offer it, it kind of requires it. 

Firstly, speed-bump or no speed-bump, the race to displayed orders, and some non-displayed orders, on IEX is a latency race and what counts is you triggering first. Fastest mostly wins, even on IEX. To hit first you need to be as close as possible to the POP. The POP is in a Equinix data centre, NY5, at 800 Secaucus Road, Secaucus, NJ. If you wish to compete for a trade you need to be a nanosecond, or picosecond, faster than your competition. To do that, the obvious way is to co-locate, which means getting space in NY5. So much for no co-location. Strike one.

Co-location is actually one of the great levellers in financial markets. No longer do you have to find where the matching engine is and then secrete a location and comms link as close as possible. Co-location makes trading fair, to suggest otherwise is just naive of Brad. He needs to think that one through a little more.

Now, close your eyes. How are you still reading? Oh well, think of an exchange that takes an hour to process a transaction. Think of that slow exchange competing against an exchange that executes in a microsecond. Imagine an external stimulus, such as a non-farm payrolls announcement of one million jobs created for the month. Follow the trades. The price action is going to be pronounced on the microsecond exchange and who knows what the hell is happening to your orders on the hour delayed exchange. Fast exchanges allowing hedging, liquidity replenishment, efficient price discovery, and are an obviously better solution. 

In the real world, some exchanges operate under 100 microseconds in round trip times (RTTs), some lower than 40 micros. IEX, due to the speed bump, is at least 700 micros, with 350 micros of delay in, plus processing, and then 350 micros of delay out. On a 40 microsecond exchange you may be able to do twenty transactions to minimise your risk, hedge, replenish liquidity, whatever, whilst you live in the land of the unknown at IEX with risk resplendent. Slow exchanges are dumb. IEX is slow.

There is a good chance speed-bumps may proliferate when IEX is approved. Speed-bumps are dumb but clients think they benefit from them, thanks to Lewis and IEX. Exchanges will find ways to give clients what they want. What a mess...

The Bad - complex orders

There was a twitter mention of IEX having simple orders and pricing. I've suggested earlier that DPEG is complex, so hearing about the simplicity of IEX grates a little, so I challenged for someone to come us with a simple explanation for DPEG. See below:
That is not a bad attempt, but not only does it over-simplify, that is if you understand the terminology, it is wrong. One of the clever things you might do with a DPEG is set a limit inside the mid-point so you can use the last-look feature to maintain a passive price near the NBBO that doesn't get adversely selected. Sal forgot about that bit. DPEG is a great order type for an HFT. 

Part of the problem with all of this is the original promise IEX made: it would keeps the orders simple. In Flash Boys, Lewis talks about how the "Puzzle Masters" reviewed some 150 different order types at various exchanges and they were aghast at complexity of it all. Here is an extract (click to enlarge):

Flash Boys quote (click to enlarge)
OK. Pages of specifications? Nuances in understanding? Let's have a detailed look at IEX's patent pending DPEG:

An overview of a Pegged Order from the IEX rules:

(Click to enlarge)

That is straight forward enough. Non-displayed (aka dark), mid-point, etc. Let's look at the DPEG specifics.
(click to enlarge)
OK. Getting a bit more complex. Priority, the less aggressive of..., hold it, what is this "except during periods of quote instability." Let's have a look at that:

(click to enlarge)
Hmmm, it doesn't say it here, but DPEG uses undelayed data, that is non-speed bumped data, to look at the current outside market and the market a second ago, that is the equivalent of looking 650 microseconds back into the past and 350 microseconds into the future then using this formula:

Nice formula.

Not complex at all, right? Though remember it is really 350 microseconds into the future, and 650 microseconds into the past, not one millisecond, due to the undelayed market data feed usage.

There are some further considerations regarding your order as a non-displayed order too:
 Now IEX supports an anti-internalisation group identifier:

There are some more details on the practice of Price Sliding, but instead of dumping the rules here, I've just pulled out an explanatory bit for you:

There's more, but let's stop now. Do you agree, DPEG is not a simple order type? This goes against the very premise of IEX given in the "Puzzle Masters" section of Flash Boys.

Order type complexity is one of the few real concerns appropriately raised in Flash Boys. A valid point in a sea of trash. Order type proliferation is a real problem, as Haim Bodek will tell you quite correctly.  With scores of order types at dozens of venues how the hell is a investor meant to understand how to best trade? Now, you can see how IEX has fallen into the trap of order complexity just as many other exchanges also have. It is a slippery slope. Order types are nearly always invented to meet a perceived or real need. There is a real potential benefit to the market maker with a DPEG that makes it a useful order type even if it does come at the cost of other types of investors, such as institutions trying to take liquidity, like Brad tried at RBC.

The Bad - Price complexity

IEX said it would stick to simple pricing. No maker / taker. Brad and Lewis went over how maker / taker pricing was bad in the book. It was deemed a lecherous evil that should be banished. IEX would have simple pricing of 9 mils for all sides. The anti-rebate argument is crap, as I've written about previously, "Trading rebates - a choice, not an evil.

That makes IEX one of the most expensive ways to transact in NMS with an 18 mil round trip. If IEX could pull that off, it would also make IEX one of the richest exchanges by gouging their customers on all those round trips. Some maker / maker exchanges only make a mill or two from the passive / aggressive offset. I don't know of any that pilfer 18 mils. Other exchanges will rebate 25 and charge 26 for a nett of 1, or some such. That is, IEX is around an order of magnitude more expensive to trade at than, say, BATS.

Well, single point pricing didn't last long at IEX anyway. They currently have a zero price holiday for displayed orders that has been extended till the end of July, but not quite all displayed orders. Let's look at how you can work it out. Originally, you had to be an approved subscriber before April 1, 2015, now it applies to all subscribers. 

This seems simple enough, 
"Trades of shares displayed on the IEX Book are charged a fee of 0 mils/share for both the adder and remover of those displayed shares during the promotional pricing period."
However, what if you go to take best, but you hit a non-displayed order? Well, you get price improvement, but now you may have to pay a fee because, 
"Effective June 1, 2015 (previously announced in Trading Alert #2015-011) and during the promotional pricing period, on a monthly basis, Subscribers who execute displayable interest against non-displayed orders, which would have incurred the standard fee, are eligible for the pricing promotion on such executions; provided that at least 90% of their aggregate executions of displayable orders added liquidity during such month."
So 0 if a displayed order, except if I hit a non-displayed order, unless 90% of my aggregate executions have added liquidity for the month then it is 0, otherwise it's 9, unless I hit an internalised order as all of those are free but not on the exchange as internalised orders have been dropped as part of the exchange transition, unless my order gets routed away and then I pay ....

It's all good, as the exchange will tell you, 
"Trade Liquidity Indicator" (FIX tag 9730) can be used by Subscribers to determine if an execution at IEX involved displayed or non-displayed liquidity. This tag is generally used in conjunction with "LastMkt" (FIX tag 30) to help calculate trading fees. 
Effective April 1, 2015, Trade Liquidity Indicator (FIX tag 9730) was expanded to support the following values when an order is executed at IEX: 
S = a self-matched trade which did not involve displayed liquidity
SL = a self-matched trade which did involve displayed liquidity 
I = a trade on IEX which neither self-matched nor involved displayed liquidity 
L = a trade on IEX which did involve displayed liquidity but did not self-match 
IEX conventions for routable orders will remain unchanged. If a routable order is submitted to IEX and ultimately executed at an external destination, IEX will pass back the Trade Liquidity Indicator (FIX tag 9730) defined by the executing venue to the Subscriber.
For reference, the Last Liquidity Indicator (FIX tag 851) can be used to determine whether the trade in question added (value 1) or removed (values 2 or 9) liquidity on IEX.
Easy enough, except for the bit about keeping track of your 90% of aggregate executions and adding liquidity. 

It's simpler than many exchanges that have volume breaks, makers benefits, etc, but it is not the one size fits all as originally promised, is it? Zero makes sense when you can't eat humble pie and rebate, but it is not quite as planned.

Remember DPEG, and perhaps other Peg, non-displayed orders are a big part of the volume picture, and rationale for IEX. You pay for non-displayed. That is a lot of expensive trading that is taking place.

The Bad - SIP gaming

Someone may know about your order being filled or cancelled before you do. In this way IEX encourages the latency arbitrage of their own investors' orders. Yep, you read that right. 

IEX will report to the SIP undelayed notifications as is required by Reg NMS. At least that is what is in their rule book (pp 206): 
(B) Securities Information Processors. Pursuant to IEX Rule 11.240(c) and IEX Rule 11.240 (d), the System connects to the SIPs to disseminate quotation and last sale (i.e. execution) information. Communications with the SIPs do not traverse the POP.
This may not have mattered too much not that long ago when the SIP was as slow as old dog with a limp during a hot day on a rough outback road, but now the SIP may sometimes be a little faster than the 350 microseconds shoebox delay. This quickly gets ugly, so let's put this one in the ugly basket as it is not so relevant to IEX, the dark pool or ATS.

The Ugly - SIP gaming

Let's have a quick look at where IEX is in NJ:
IEX Pop (click to enlarge)
In the above picture, you can see NY5 is marked in red at 800 Secausus Road which is the IEX Pop. Everything to and from IEX goes through the Pop. To compete effectively you need to co-locate at NY5. NY5 is an Equinix DC. One very small yellow star, very small I know sorry, is south west for NY2 at 275 Hartz Way. Another very small yellow star is south easterly at 755 Secaucus Road and this is NY4. It's a busy little exchange area. BATS split their DC's between NY4 and NY5 in a latency equalised way which is pretty simple as there is only a couple of microseconds, if that, between the DCs by fibre. Remember every 200m in fibre is a microsecond.

The actual matching engine for IEX was located at 1919 Park Avenue, Weehawken:

IEX matching engine (click to enlarge)
The yellow star to the north east of the IEX marker in the map is 300 Boulevard East, NJ2, and the reference point we'll use shortly.

Between NJ2 in Weehawken and NY5 there is about 4.1 km of distance as the fibre crow would fly if it could, but it doesn't. That would equate to about 41 fibre microseconds of round trip time (RTT) if that were possible, which would be a minimum possible optical fibre time. That is a bit less than the 700 microseconds RTT enforced by the magic shoebox, but it is also not clear if that is plus 700 micros in additional to normal overheads or equalised to 700 micros.

Now, fibre is 5 microseconds per km, but the speed of light, or 'c', is 3.33 microseconds per km. That is fibre is 2/3c whilst RF, such as light, microwave, and millimetre wave travels at ~1c, or 50% faster. So it would be about 27.4 micros RTT between IEX's pop and its matching engine by RF.

Now, IEX reports undelayed fills to the SIP. I presume this goes from the Weehawken, but it could be Secaucus. There are two SIPS: one run by NYSE for Tape A and Tape B, the CTS; and, one run by Nasdaq for Tape C, i.e. UTP. The NYSE SIP is at 1700 MacArthur Boulevard, Mahwah in the NYSE liquidity centre and the Nasdaq SIP is down in Carteret in the Verizon DC at 1400 Federal Boulevard with the rest of the Nasdaq operations. Let's have a look at these on a map of NJ.

Mahwah/NYSE, to the North; Carteret/Nasdaq is Southwest. IEX Pop & Match in the middle
(click to enlarge)
Here is a simplified version of the map, care of McKay Brothers:

(click to enlarge)
The potential hole in IEX's model here is that the undelayed reporting to the SIP maybe faster than getting the data directly via the magic shoe box delay at the Pop. Could this be the case?

A few years ago, the answer was definitely NO as the SIP overhead was measured in milliseconds. Let's look at the SIP latencies now:

For UTP, you'll find whilst the average latency has dropped from 2010's 5,420 micros to 920 micros, the 10th percentile latency of 390 microseconds makes it, once you add in travel time, unlikely you'll benefit over a competitor waiting diligently at the IEX Pop in Secaucus.

For the CTA SIP, the average latency overhead has dropped from 11,450 micros in 3Q2010 to 490 micros now which would be discouraging if latency distribution was not helpful, but it is. The Median latency is 220 microseconds, and the 10th percentile latency is 150 microseconds. Depending on the travel times, there may be a way to take advantage of this at IEX. Let meander through this some more.

Travel times

Not many people will give us credible travel times, so lets look at the distances, implied minimum fibre times, implied minimum RF times, and whatever public data for wireless we can find:

Distance – km Mahwah NY4 NY5 NJ2 Carteret
Mahwah - 34.0 34.3 36.8 55.6
NY4 34.0 - .3 4.1 25.9
NY5 34.3 .3 - 4.3 26.1
NJ2 36.8 4.1 4.3 - 27.0
Carteret 55.6 25.9 26.1 27.0 -

Min: RF – uWave/mWave/Laser Mahwah NY4 NY5 NJ2 Carteret
Mahwah - 113.3 114.3 122.5 185.4
NY4 113.3 - .9 13.7 86.3
NY5 114.3 .9 - 14.4 86.8
NJ2 122.5 13.7 14.4 - 89.9
Carteret 185.4 86.3 86.8 89.9 -

Min: Optical Fibre Mahwah NY4 NY5 NJ2 Carteret
Mahwah - 170.0 171.5 183.8 278.2
NY4 170.0 - 1.4 20.5 129.5
NY5 171.5 1.4 - 21.7 130.3
NJ2 183.8 20.5 21.7 - 134.9
Carteret 278.2 129.5 130.3 134.9 -

TMX – radio/radio Mahwah NY4 NY5 NJ2 Carteret
Mahwah - 118.0 - - 192.5
NY4 118.0 - - - 92.0
NY5 - - - - -
NJ2 - - - - -
Carteret 192.5 92.0 - - -

TMX – rack/rack Mahwah NY4 NY5 NJ2 Carteret
Mahwah - 124.5 - - 202.0
NY4 124.5 - - - 100.0
NY5 - - - - -
NJ2 - - - - -
Carteret 202.0 100.0 - - -

Quincy Data Mahwah NY4 NY5 NJ2 Carteret
Mahwah - - - - -
NY4 - - - - 96.0
NY5 - - - - -
NJ2 - - - - -
Carteret - 96.0 - - -

Nasdaq – Apsara Mahwah NY4 NY5 NJ2 Carteret
Mahwah - 117.5 - - 192.0
NY4 117.5 - - - 90.0
NY5 - - - - -
NJ2 - - - - 98.0
Carteret 192.0 90.0 - 98.0 -

Spread Networks Mahwah NY4 NY5 NJ2 Carteret
Mahwah - - - - -
NY4 - - - - 95.0
NY5 - - - - -
NJ2 - - - - -
Carteret - 95.0 - - -

OK. We can see the IEX engine at NJ2 to CTA SIP at Mahwah distance is 36.8 km. This could be one or two hundred metres out depending on roof placements (they are big buildings), etc. This translates to over 183.8 fibre microseconds (one way), or over 122.5 RF microseconds (one way). Even if we had RF both ways from IEX to the SIP we're not going to get an advantage as 114.3 + 122.5 + 150 = 386.3 > the pop's 350. It's close and perhaps at the 1st percentile, or if the SIP advances there could be something in it, eventually, if the SIP was really fast RF internally.

Not all is lost. There is a chance that if we can do a latency arb to NYSE or another service in Mahwah. We could conceivably advantage ourselves over the poor guy waiting for his fill back at IEX. The best case, that is somewhat unlikely:

  • NYSE RF only path = 122.5 + 150 = 272.5
  • IEX path = 350 + 114.3  = 464.3
It shows that using the SIP in Mahwah has the potential to be faster than the investor at IEX using RF to get to Mahwah to potentially trade.

Let's assume a not so rosy scenario, that is, the path to Mahwah is an optical one:
  • NYSE optical path = 183.8 + 150 = 333.8
                                                           < 464.3!
At 334 micros we still have some room to take up some of the likely inefficiency and still be faster than the trader at IEX wanting to do an arbitrage at NYSE via RF.

If we look at real world RF latencies reported, we see about 4 to 15 micros of extra overhead on the links. Though the end-point to end-point specification is only sometimes crystal clear.

Naturally, I suspect the SIP communications will have significantly extra overheads, firewalls, SFTI foo, and perhaps not quite make it, but it might. 

The point here is that this is exactly what IEX is not meant to be about. If this scenario gets realistic, a trader may need to have multiple sites of trading set up at various liquidity centres to ensure that she does not get back-run, latency arb'd, or whatever you want to call it. IEX may make your trading set-up a lot messier and more expensive.

This is ugly, no?


I'm not that fussed about IEX itself as I think its business model, is, ahhh, non-optimal. Apart from some innovation with the complex, dark, DPEG order to suit an HFT seeking to avoid adverse selection, it is a pretty dumb and expensive price discovery and trade solution. Being dumb is not against Reg NMS rules though.

I still think that provided Reg NMS is not undermined, and the SEC does not allow protected status for potentially stale IEX quotes (adopt similar rules to IIROC & exclude tape fees), IEX should be allowed to be an exchange. IEX should be allowed the good grace of capitalistic failure even if that may take a while due to its wrong-headed fanboy marketing momentum. 

However, the utility of Reg NMS is at stake if other exchanges also get greenlit for delays, partial delays, more complex orders, further darkness, etc. It could be a real mess. Apart from the sad escalation in costs, and further barriers to entry, that is terrific news for us HFT types, and market structure analysts, as the IEX approval will create some very tricky market structure to take advantage of. 

Flash boys & IEX?  
  • Low and simple costs = FAIL
  • Simple order types = FAIL
  • Protected from latency arb = FAIL
  • Colocation not required = FAIL
  • Sophisticated infrastructure and microwave not required = FAIL
  • Good intentions nevertheless = hollow SUCCESS
IEX shareholders need to think long and hard about this pup they've been sold. IEX has plenty of cash. It is not too late to pivot and build an exchange with real solutions that would actually help investors.

Jane Doe on Main Street needs a different solution.