And Why There Still May be Some Free Cheese in my Future.
Occasionally I like to bet. And in Matthias Troyer I found somebody who took me up on it. I wrote about this bet a while ago, but back then I agreed that I wouldn’t identify him as my betting pal, until his paper was published. Now the paper has been out for a while and it is high time to settle the first part of this bet.
The conditions were straightforward, can the D-Wave machine beat a single classical CPU? But of course we specified things a bit more precisely.
The benchmark used is the time to find the ground state with 99% probability, and then not only the median is considered but also the 90%, 95% and 99% quantile. We then agreed on basing the bet on the 90% quantile. I.e. the test needs to run long enough to make sure that for 90% or more of the instances we find a ground state with 99%.
Assuming that Matthias gets to conduct his testing on the current and next chip generation of D-Wave, we agreed to make this a two part bet, i.e. same betting conditions for each.
Unfortunately, I have to concede the first round. The D-Wave One more or less tied the classical machine, although there were some problem instances where it was doing better. So the following jars of Maple Syrup will soon be shipped to Zürich:
What I was obviously hoping for was a decisively clear performance advantage, but at this point this isn’t the case, unless you compare it to off-the-shelf optimizer software as was done in the test conducted by McGeoch et. al.
This, despite the evidence for quantum entanglement of D-Wave’s machines getting ever more compelling. A paper has just been published in Pysical Review X, that demonstrates eight qubit entanglement. Geordie blogged about it here, and it already generated some great press (h/t Pierre O.), probably the most objective mainstream article on D-Wave I’ve seen yet. It is a welcome change from the drivel the BBC put out on QC in the past.
So will I ever get some Raclette cheese in return for my Maple syrup? The chances for winning the next part of my bet with Matthias hinge on the scaling behavior, as well as on the question if a class of hard problems can be identified where quantum annealing manages to find the ground state significantly faster. For the generic randomly generated problem set, scaling alone does not seem to cut it (although more data will be needed to be sure). So I am counting on D‑Wave’s ingenuity, as well as those bright minds who now get to work hands-on with the machine.
Nevertheless, Matthias is confident he’ll win the bet even at 2000 qubits. He thinks D-Wave will have to improve much more than just the calibration to outperform a single classic CPU. On the other hand, when I had the pleasure of meeting him last year in Waterloo, he readily acknowledged that it was impressive what the company had accomplished so far. After all, this is an architecture that was created within just ten years based on a shoestring budget, compared to the multi
mbillion dollar, decades mature semiconductor industry.
Unfortunately, when his university, the venerated ETH Zürich (possibly the best engineering school on this planet) came out with this press release, they nevertheless (accidentally?) played into the old canard that D-Wave falsely claimed to have produced a universal quantum computer.
It puts into context the Chinese whisper process as depicted in this cartoon that I put up in an earlier post. Unlike depicted here, where the press gets most of the blame, ever since I started paying attention to university press releases, I am compelled to notice that they are more often than not the true starting point of the distortions.