Tag Archives: Geordie Rose

The Church of D-Wave

Are You a BDeliever? dwave_churchgoer Science and religion have a difficult relationship, and sometimes they combine in the most obscure manner, such as when Scientology was conceived.  The latter seems to have lost a lot of its appeal and followers, but it seems that another new religion is poised to grab the mantle.

That is, if one is willing to follow Scott Aaronson's rationale that believing in the achievability of significant speed-up with D-Wave's architecture is a matter of faith. Ironically Scott, who is teaching computer science at MIT, made this comment about the same time that the MIT Technology Review named D-Wave to its Top 50 Smartest Companies list. An illustrious selection, that any company would be delighted to be included in. The only quibble I have with this list is that it ranks Elon Musk's SpaceX before D-Wave, my point being that quantum mechanics is harder than rocket science. After all, with the latter, everybody can decide if your spacecraft made it into orbit or not (classical mechanics is so straightforward).  On the other hand, we still have the ongoing high profile battle over the question of how quantum D-Wave's machine actually is (since Schroedinger the uncertainty of what's in a box seems to be a constant in Quantum Mechanics).

Another paper buttresses the company's claims that there is substantial entanglement present on their chip.  This prompted Prof. Vazirani, who I experienced as a most delightful soft spoken academic when checking out his Quantum Computing MOC, to come out swinging.  The New York Times quotes him as saying:

“What I think is going on here is that they didn’t model the ‘noise’ correctly. (....) One should have a little more respect with the truth.”

In academic parlance these are fighting words.  And so the show goes on.

But I want to take a break from this for a moment, and focus on another question: How did a startup like D-Wave get to this point?  Time magazine front page material, coverage in the New York Times, being named in the same breath as SpaceX.  From a business perspective this is nothing but an amazing success story to have gotten to this point. And to me, the question of what makes successful entrepreneurship is of no less interest than science and technology.

Geordie_Gold
Geordie got closer to having a shot at Olympic gold than most of us, having been an alternate on the Canadian wrestling team at the 1996 Olympic Games, so getting this one may have been bitter sweet.

Flying into Vancouver I imagined Geordie Rose to be a Steve Jobs-like character, about whom it was famously quipped that he was surrounded by his own reality distortion field, an invisible force that made others see the world like he did, and made them buy into his vision. And although I never had the pleasure of meeting Steve Jobs, I think it is safe to say that Geordie is nothing like him. If I had to describe him in one word, I'd say he is quintessentially "Canadian", in terms of the positive attributes that we typically like to associate with our national character. (Full disclaimer: Technically I am not Canadian yet, just a permanent resident).

Given the amazing success that D-Wave has had, and the awards and accolades that he himself has received, I was impressed with his unassuming demeanor. Hard to imagine Geordie would ever park his car in a handicap spot, as Jobs was fond of doing, to shave a couple minutes off his commute.

D-Wave just moved to a new enlarged premises. In their old building Geordie occupied an interior office without windows. I naturally assumed that he would have upgraded that. So I was surprised to learn that his new workspace still doesn't have any windows. His explanation was simple, it allows him to be close to his team.

My take away is that visionaries cannot be pigeon-holed, because when talking to Geordie it was quickly obvious that his focus and dedication to making his vision a reality is ironclad, and his excitement is infectious.  So this is one key similarity to Steve Jobs after all, and then there is of course this, which goes without saying:

Great entrepreneurs never do it for the money.
Great entrepreneurs never do it for the money.

Prof. Vazirani must have picked up on D-Wave's commitment to make Quantum Computing work, as the New York Times also quotes him as saying about D‑Wave that “after talking with them I feel a lot better about them. They are working hard to prove quantum computing.

That Geordie picked an approach which is so abhorred by theorists, I attribute to yet another aspect that, in my mind, marks great entrepreneurship: An almost ruthless pragmatism. Focusing on the less proven quantum annealing on a chip, he managed in just seven years to turn out an entirely new computing platform.  Meanwhile, the advances in superconducting foundry know-how that his company ushered in, will also benefit other approaches, such as the gate based implementation that UCSB's John Martinis plans to scale up to 1000 qubits within five years.

To me, there is no doubt that the hurry to get something to the market is a net benefit to the entire quantum computing field, as I expect it will attract more private capital. And that is because Quantum Computing is now no longer perceived as something nebulous, something that just may happen 25 years down the road.

Game changers polarize.  So if we pay heed to Scott Aaronson's rhetorics Geordie clearly has a leg up over Steve Jobs.  Where the latter had a cult following, Geordie's on his way to having his own religion.  Maybe that'll explain the following recent exchange on D-Wave's blog:

D-Wave_blog

 

(h/t Rolf D. and commenter Copenhagen for pointing me to material for this post.)

Out of the AI Winter and into the Cold

dwave_log_temp_scale
A logarithmic scale doesn't have the appropriate visual impact to convey how extraordinarily cold 20mK is.

Any quantum computer using superconducting Josephson junctions will have to be operated at extremely low temperatures. The D-Wave machine, for instance, runs at about 20 mK, which is much colder than anything in nature (including deep space). A logarithmic scale like the chart to the right, while technically correct, doesn't really do this justice.  This animated one from D-Wave's blog shows this much more drastically when scaled linearly (the first link goes to an SVG file that should play in all modern browsers, but it takes ten seconds to get started).

Given that D-Wave's most prominent use case is the field of machine learning, a casual observer may be misled to think that the term "AI winter" refers to the propensity of artificial neural networks to blossom in this frigid environment. But what the term actually stands for is the brutal hype cycle that ravaged this field of computer science.

One of the original first casualties of the collapse of artificial intelligence research in 1969 was the ancestor of the kind of learning algorithms that are now often implemented on D-Wave's machines. This incident is referred to as the XOR affair, and the story that circulates goes like this:  "Marvin Minsky, being a proponent of structured AI, killed off the connectionism approach when he co-authored the now classic tome, Perceptrons. This was accomplished by mathematically proving that a single layer perceptron is so limited it cannot even be used (or trained for that matter) to emulate an XOR gate. Although this does not hold for multi-layer perceptrons, his word was taken as gospel, and smothered this promising field in its infancy."

Marvin Minsky begs to differ, and argues that he of course knew about the capabilities of artificial neural networks with more than one layer, and that if anything, only the proof that working with local neurons comes at the cost of some universality should have had any bearing.  It seems impossible to untangle the exact dynamics that led to this most unfortunate AI winter, yet in hindsight it seems completely misguided and avoidable, given that a learning algorithm (Backpropagation) that allowed for the efficient training of multi-layer perceptrons had already been published a year prior, but at the time it received very little attention.

The fact is, after Perceptrons was published, symbolic AI flourished and connectionism was almost dead for a decade. Given what the authors wrote in the forward to the revised 1989 edition, there is little doubt how Minsky felt about this:

"Some readers may be shocked to hear it said that little of significance has happened in this field [since the first edition twenty year earlier]. Have not perceptron-like networks under the new name connectionism - become a major subject of discussion at gatherings of psychologists and computer scientists? Has not there been a "connectionist revolution?" Certainly yes, in that there is a great deal of interest and discussion. Possibly yes, in the sense that discoveries have been made that may, in time, turn out to be of fundamental importance. But certainly no, in that there has been little clear-cut change in the conceptual basis of the field. The issues that give rise to excitement today seem much the same as those that were responsible for previous rounds of excitement. The issues that were then obscure remain obscure today because no one yet knows how to tell which of the present discoveries are fundamental and which are superficial. Our position remains what it was when we wrote the book: We believe this realm of work to be immensely important and rich, but we expect its growth to require a degree of critical analysis that its more romantic advocates have always been reluctant to pursue - perhaps because the spirit of connectionism seems itself to go somewhat against the grain of analytic rigor." [Emphasis added by the blog author]

When fast-forwarding to 2013 and the reception that D-Wave receives from some academic quarters, this feels like deja-vu all over again. Geordie Rose, founder and current CTO of D-Wave, unabashedly muses about spiritual machines, although he doesn't strike me as a particularly romantic fellow. But he is very interested in using his amazing hardware to make for better machine learning, very much in "the spirit of connectionism".  He published an excellent mini-series on this at D-Wave's blog (part 1, 2, 3, 4, 5, 6, 7).  It would be interesting to learn if Minsky was to find fault with the analytic rigor on display here (He is now 86 but I hope he is still going as strong as ten years ago when this TED talk was recorded).

So, if we cast Geordie in the role of the 21st century version of Frank Rosenblatt (the inventor of the original perceptron) then we surely must pick Scott Aaronson as the modern day version of Marvin Minsky.  Only that the argument this time is not about AI, but how 'quantum' D-Wave's device truly is.  The argument feels very similar: On one side, the theoretical computer scientist, equipped with boat-loads of mathematical rigor, strongly prefers the gate model of quantum computing. On the other one, the pragmatist, whose focus is to build something usable within the constraints of what chip foundries can produce at this time.

But the ultimate irony, it seems, at least in Scott Aaronson's mind, is that the AI winter is the ultimate parable of warning to make his case (as was pointed out by an anonymous poster to his blog).  I.e. he thinks the D-Wave marketing hype can be equated to the over-promises of AI research in the past. Scott fears that if the company cannot deliver, the babe (I.e. Quantum Computing) will be thrown out with the bathwater, and so he blogged:

“I predict that the very same people now hyping D-Wave will turn around and—without the slightest acknowledgment of error on their part—declare that the entire field of quantum computing has now been unmasked as a mirage, a scam, and a chimera.”

A statement that of course didn't go unchallenged in the comment section (Scott's exemplary in allowing this kind of frankness on his blog).

I don't pretend to have any deeper conclusion to draw from these observations, and will leave it at this sobering thought: While we expect science to be conducted in an eminently rational fashion, history gives ample examples of how the progress of science happens in fits and starts and is anything but rational.