The other argument that Flynn plays out is that compiler assisted parallel coding is challenging and typically has not demonstrated scaling parallel speed ups beyond 8 or so conventional cores. Again fine, fits with my experience from SGI compilers to now, I have no issue. Parallel programming with pragmas is no way to write code for fun and profit.

The jump to “so therefore dataflow is good for a bunch of general purpose mathmatics” needs further justification, maybe it is I don’t know. In particular financial applications like: closed form expression evaluation, discounting, lattices, and Monte Carlo for big portfolios of trades/positions, all hit the memory hierarchy slightly differently. The vectorization opportunities also vary in degree between these financial mathematics applications. I’m not sure about this but I think there were science experiments conducted somewhere uptown that showed you can teach dolphins PERL and Python and they can parallelize these position inventory calculations to speed, just don’t let them try to do load balancing because that will delay the project.

Perhaps Dataflow has some large performance advantage in 2012 but the costs of converting to FPGAs or waiting for the latest FGPA Xilinx parts in your supercomputer more than compensates for the advantage. Certainly you do not expect a massive popular dataflow movement where all your friends, holding copies of Dataflow for Dummies, flock about you to find out how to program dataflow on their phone apps. That’s not happening. Best case you are going to get a cool dataflow programming result that will make you a hero in the dataflow community but as a reward you will have to simply smile inwardly to yourself knowingly, because really no one else will ever know what you did or what you are talking about.  What about debugging the production infrastructure and code, doesn’t that  kill you inside a little bit every time you  think about it? So, maybe it’s just me but this Dataflow architecture performance advantage better be pretty big to account for all the obvious really bad stuff.

Taking Flynn’s argument on face value I would guess there is cross over time for Dataflow architecture to maintain a significant performance advantage over off-the-shelf architecture, assuming everything else stays the same. I would think the cross over time will be closer to the time we see volume production of 16 multicore microprocessor chips, 2015?  But that assumes nothing else  changes. Big assumption.

From a Credit trading perspective, realtime or otherwise, there is nothing to see here, just move on.

From a macro risk perspective it is unambiguously a good thing to have a fast view of  the aggregate risk across single name, index, and correlation inventory. Getting solid analytic numbers for a old piece of quant code is notoriously hard. No one with actual market background will check market convention assumptions, trade representations , or the valuations.  If some enterprising company  has a tied out version of the old quant code get it from them.

But there is more to this story. If the broker dealer cannot bring themselves to separate the correlation book from the credit book arguing that the correlation book is managed to zero risk and standard reserves are held  while the trades roll off. Then in order to run second order risks like counterparty valuation adjustment, the fast correlation risk becomes important. Unless you get the correlation book running to speed your second order risk monte carlo simulation batches performance will be dominated by the correlation book performance (assuming every other product book batch demonstrates competitive performance and better turnaround time compared to the correlation book). Hello, Maxeler.

The Economist defends the Gaussian Copula, here.

Zerohedge:  Hour in the life of CDX trading in Europe, here. One of a kind piece and exceptionally worth your time to read.

Basic explanation of CDOs from RGE monitor, here  - Note the age of these links, in credit this stuff has not traded  in any sort volume for many years. The CDO links are useful for getting context around the JPM credit batch performance. In Credit, learning Gaussian Copula in 2012 is kind of like learning how a record store works in 2012.

Markit 2008 Primer on Credit Indicies, here.

Creditflux.com, here –  Creditflux is the leading information source globally for credit trading and investing, credit derivatives, structured credit, distressed credit and credit research, publishing a monthly newsletter, online daily news and a comprehensive database of CDOs and credit hedge funds.

Kurzweil is very motivated to stay alive here is his website.

Delivering the next generation of supercomputing with reconfigurable hardware video O. Mencer, here.

League tables for CDO arrangers 2011, here. JPM did one CDO/CLO in Europe and one in US in 2011 according to these folks at Leading Structured Finance News. I don’t believe the other structures are going to be handled in the credit batch.  Maybe there is a lot of prop trading  in standard tranches that needs realtime correlation risk -  let’s look. Need to find global CDO issuance in 2010 and 2011 but here is 2005 through 2009. You can see why fast Gaussian Copula would have been great between 2005 and 2007.

By Anh Nguyen | Computerworld UK | Published 11:01, 15 December 11

here

…

The supercomputer solution was developed with HPC solutions provider Maxeler Technologies, which is also providing JP Morgan with the new, upgraded supercomputer for additional areas of the business.

The new dataflow supercomputer – where the computer chips are tailored to perform specific, bespoke tasks – will be equivalent to more than 12,000 conventional x86 cores, providing 128 Teraflops of performance.

Zerohedge on the CDS market trade volume distribution and a proposal for CDS indicies to be exchange traded, here.

Kamakura Corporation, Risk management tools and Jarrow is involved somehow look at the research and blogs, here.

Cloud Computing get reviewed by US DOE, Argonne, and Lawrence Berkeley and gets a grade of meh, here from Clusterstock.

Salmon  on Udacity, here.  Stanford AI professor starts up online University UDACITY. Agreed this looks like it could grow. An AI course at Stanford gets 100K worldwide enrollment? wow. I would like to hear  why the notion that Stanford, Harvard, Ptown, or Oxford should brand this is an obviously bad idea.

HPCWire Russell Fish @ Venray Technologies has an embedded Microprocessor in DRAM  play, here. Problem is apart from Mortgages I doubt much Street P&L/Risk analytics is intrinsically memory bandwidth starved as opposed to processing starved. Super good at creating pipeline bubbles though.

HPCWire – here  J.P. Morgan Deploys Maxeler Dataflow Supercomputer for Fixed Income Trading

Peter Cherasia, Head of Markets Strategies at J.P. Morgan, commented: “With the new Maxeler technology, J.P. Morgan’s trading businesses can now compute orders of magnitude more quickly, making it possible to improve our understanding and control of the profile of our complex trading risk.”

insideHPC – here  JP Morgan Fires Up Maxeler FPGA Super

Compute Scotland – here Near realtime: JP Morgan & Maxeler

Oh and Terry Tao on Black Scholes, here

“Sandy Bridge-EP” Xeon E5 processors and their related “Romley” server platforms, are now in volume shipment, here

Overclocking insurance for Sandy Bridge from Intel, here

Whoa more to think about now: Maxeler says Intel’s Knights Ferry simplicity might not suit HPC, here, at The Inquirer. The article by Lawrence Latif has a  subtitle that reads “More effort yields better performance”!  I think I like the Inquirer it looks like a fancy version of the old Microprocessor Reports.

Check out the Thalesian Seminar: Stein from Bloomberg talking about CVA at the NY public Library 31 Jan 6pm

FPGAs in HFT Thalesian Seminar Slides from eigen.systems

Intel Xeon E5-2690 Sandy Bridge-EP Performance Revealed, Tom’s Hardware, here

MJ Flynn Accelerating computation with FPGAs, slides, @Berkeley video nice talk audio starts to break up around 31:00 though

Maxeler Technologies, home page

JP Morgan FPGA Careers factsheet. So, that JP Morgan credit derivative batch @300K CDS and 30K credit curves is going to run in 2 milliseconds (1000x the cheapest MacPro at the Apple Store)  but I might only get 200x in practice so it will be between 2-10 milliseconds on the 40-node hybrid FPGA machine. Do I get all Virtex6 chips or is that hybrid as well?

Xilinx Virtex-7 FPGA Family looks to be the fastest 2X and 50% lower power then there is Virtex-6.

A. Rates, Mortgage, FX, Commodities P&L/Risk batches

It’s only the credit derivative guys in the Computerworld UK article that had this P&L and Risk batch performance problem in 2008 and now it is fixed?

B. Moore’s Law

“The number of transistors incorporated in a chip will approximately double every 24 months.”  —Gordon Moore, Intel Co-Founder

More of a prediction than a law really, but it has predicted accurately for 40 years and is expected to hold for another decade.  In technology if you do nothing more than “ride the wave” of Moore’s Law you are doing good, however, this observation’s simplicity can be deceiving. If the plan is for the credit derivative batch to “ride the wave” with thousands of grid CPUs, FPGAs, and associated staff you may have a challenge when scaling to larger problems, like counterparty valuation adjustment (CVA), that Moore’s Law will not automatically address for you. For example, Moore’s Law doesn’t get you power and machine room space for 10,000 grid CPUs for the CVA simulation.   That is closer to “not riding the wave.”

C. Simplicity

Supercomputers and FPGAs, in addition to GPUs, Cuda, and CPU grids even dataflow architecture are all reasonable technology bets that could very well be the best way to ride the Moore’s Law wave through 2020. Using some or all of these technologies together in 2011 to run such a modest computation so slowly should elicit some follow up questions.

Tyler Cowen: Simulations and the Fermi paradox, here

Superlinear convergence of the Secant Method, here.

Freeman Dyson interview, here.

Fast FFT in MIT News, here.

Felix Salmon on Fact Checking, here. Does the previous Credit Derivative post get classified as fact checking?