A closeup of the “Pluto Switch,” a mystery hardware device that landed on the edge of Iowa this past winter. Images: networking-forum.com
Photos of the mystery computing device appeared on the web in late February. Taken with a smartphone, they were a bit washed out and a little blurry in places, but you could easily read the name printed on the long, thin piece of hardware. “Pluto Switch,” the label said.
The images were posted by two men who said the device had unexpectedly turned up at a branch office in the tiny farmland town of Shelby, Iowa — population: 641 — and they were hoping someone could tell them what it was.
Clearly, these two men were familiar with the ins and outs of computer networking, and clearly, this was a networking switch, a way of shuttling data between machines. But they’d never heard of the Pluto Switch, and it was littered with networking ports they’d never seen before. “Any ideas?” they asked. “The writing on the back is Finnish.”
According to posts they made to an obscure web discussion forum dedicated to networking hardware — networking-forum.com — they couldn’t actually get the thing to work. But they turned up a few clues indicating who the device belonged to, and eventually, after putting two and two together, they said they’d located the owner and sent the switch back.
It belonged, they said, to Google.
At first, Google didn’t respond to their phone calls, the men said, and when it did, it wouldn’t explain the switch. But apparently, the company offered a reward for its return. “Finally got a hold of a Google network engineer, so the switches are heading home. He wouldn’t tell me what the connector type was so that’s still a mystery,” one of the men told the forum. “The engineer was cool and is going to send us some shirts the public can’t buy.”
Google calls itself one of the world’s largest hardware makers. For the past 10 years, the web giant has designed much of the gear driving the massive data centers that underpin its web empire, but it treats the particulars of this hardware operation as the most important of trade secrets. That’s why the Pluto Switch is so intriguing.
The mystery switch may provide a small window into the networking hardware that drives the Googlenet — and indicate where the rest of the web is going. In recent years, Google’s efforts to redesign data center hardware have nudged other web giants in a similar direction, with Amazon, Facebook, and even Microsoft exploring custom-built data center gear. When a web empire reaches a certain size, you see, it needs gear that’s much cheaper and more efficient than the hardware typically offered by big-name sellers such as Cisco, HP, and Dell.
Google declined to comment on the switch. But according to J.R. Rivers — an ex-Google engineer who helped design Google’s first networking switches in 2005 — the Pluto Switch seems to run Google software, and it uses hardware you don’t typically find on standard networking switches. Though he has no way of knowing whether it was designed by Google itself, he says it bears all the signs of gear that was custom built for a massive online operation.
“I’d venture to bet that this is home-grown data center gear,” he told Wired, after reviewing the photos.
Certainly, all signs point to Google, and at the very least, the photos highlight why Google has gone to such great lengths to build its own data center hardware. According to Rivers, the switch is a significantly streamlined version of what you might buy on the open market. “The big difference,” he says, “is that this switch has forgone the standard physical layer for something less expensive.” Even those mystery networking ports are a way of cutting costs.
A glimpse of the network traffic generated by the Pluto Switch, thanks to an analysis tool called Wireshark
Pluto Lands In Shelby
The two men who posted the photos said they worked for a company that’s headquartered in Wisconsin but runs a distribution center in Shelby, Iowa, about 30 miles from a data center Google operates in another Iowa town called Council Bluffs. Apparently, the switches were delivered to the distribution center in late December or January, several weeks before the photos were posted.
“My guess is the carrier unloaded them by mistake,” said one of the two men, who seemed to be computer gurus working at the company’s HQ. “About a month later, they were shipped up to me at the corporate offices in Wisconsin. We’ve been trying to figure out what they were for the last few weeks.”
When they posted the first photo, they already had one pretty good clue on their hands. A vendor code used by the switch, they said, was registered to Google. But they didn’t seem to realize that Google designed its own switches. Eventually, others on the forum pointed them in the right direction, and the clues kept coming.
‘They said that the switches had been returned to Google and that Google had asked them to attempt to remove the thread.’
— Steve Spangle
At one point, someone else on the forum offered to buy the switches. But then he had second thoughts. “It’s very likely that these really are Google’s and I’m sure they want them back,” he said. “Apple’s lost then sold iPhone ordeal with Gizmodo comes to mind.”
At first, the two men couldn’t get into the switch “console,” the interface that would provide more info about the device. It pumped out nothing but gibberish. But when they finally got it working, others on the forum were confident the mystery was solved. The console identified the hardware board inside the device as a “Google Planet8541 Pluto Edge Switch.”
The identity of the two men has now been scrubbed from the forum. According to Steve Spangle, the owner of networking-forum.com, they requested anonymity after talking to Google. “[They] asked me to delete the thread,” Spangle tells Wired. “They said that the switches had been returned to Google and that Google had asked them to attempt to remove the thread. Instead, I scrubbed their usernames from all posts, replacing them with myself as the post author.”
The men who posted the photos may work for a company called Menard’s, a retail outfit headquartered in Eau Claire, Wisconsin that runs a distribution center in Shelby. But an employee at the distribution center said that Menard’s was not the company in question, and we were unable to reach the CIO’s office in Wisconsin. A company spokesperson did not provide answers to our questions.
Spangle says that he has not been directly contacted by Google about the posts, but that he would remove the thread if Google asked him.
As we published this story, the photos, a data dump from the console, and other information about the switch were still available on the forum. All this provides a rather detailed picture of the Pluto Switch. And it may point to much more.
Networking ports on the Pluto Switch were unlike any the forum had seen.
‘I Know Who Made This … Just by Looking at It’
When we showed the photos to Andrew Feldman — the founder of SeaMicro, a new-age server outfit now owned by chip designer AMD — he said he knows the engineer who built it. “I know who made this … just by looking at it,” he told us over the phone. “I know the engineer. I’m looking at it, and I know an engineer’s work.”
He wouldn’t say who the engineer was, and he didn’t link the switch back to Google. But Feldman has a history with Google. He once worked for Force10 Networks, a company he says sold hundreds of millions of dollars in networking equipment to Google — before Google started designing its own gear.
When we showed the photos to Google, the company said they were old. “I don’t think we have much to say on this front,” a Google spokesperson told us. “These forum posts are from over 7 months ago.”
‘I know who made this just by looking at it. I know the engineer. I’m looking at it and I know an engineer’s work.’
— Andrew Feldman
Nowadays, Google designs most of the hardware that drives its web services, including servers and the data center facilities themselves, and it sees this custom hardware work is a competitive advantage over all the other big-name websites struggling to instantly accommodate requests from millions of users across the globe. That’s why it treats the hardware as a trade secret.
Earlier this year, after years of rumors, Google revealed that it designs specialized networking hardware for moving data between its data centers — hardware that uses a new protocol for managing networking gear called OpenFlow — but the company provided few details about the hardware itself. And it’s still tight-lipped about the networking hardware used to move data inside its computing facilities. As recently as June, Urs Hölzle — the man who oversees the Googlenet — declined to discuss the matter.
But the information posted to the forum indicates that the Pluto Switch is at least akin to devices Google uses inside its data centers. An “edge switch” is a device that connects servers inside a data center to a larger network. These are also called “top-of-rack” switches, as they typically sit at the top of a rack of servers.
Ex-Google engineer J.R. Rivers — who now runs a networking outfit called Cumulus Networks — says this appears to be an older switch, something that has been used for about three years or so. And he questions whether Google would put label like “Pluto Switch” on its custom-designed gear. But in all likelihood, this is a device that belongs to Google, and as described in the forum, it’s representative of the company’s overall hardware philosophy.
With its custom hardware, Google aims to improve the operation of its data centers, but it also seeks to reduce costs. Because it operates at such an enormous scale — the company now runs its services across as many as three dozen data centers across the globe — it can save vast amounts of money by reducing power consumption and stripping hardware to its bare essentials.
According to Rivers, the Pluto Switch is an example of this sort of streamlining. He points out that the “physical layer” that transports data across the device is a stripped-down version of what you’d see in a typical switch. It still uses optical fiber lines to move data, but it relies on a much less expensive collection of hardware.
Those networking ports on the Pluto had many people on the forum scratching their heads. “That must have come from outer space (as the name also implies),” said one poster on the forum, responding to the photos. But Andrew Feldman says they’re a way of cutting costs. “This is a different type of connector, because connectors are extremely expensive,” he told us. “They’re designed to take a modified cable. My guess is they’re making the cable as well.”
A look inside the Pluto Switch, where a Broadcom “Scorpion” chip drives the device’s 10 gigabit connections
Where the World Is Going
The Pluto Switch was built for a network that can theoretically handle 10 gigabits of data a second, a big step up from the 1 gigabit networks that still dominate the data center landscape. According to Rivers, it offers 20 ports for 10-gigabit speeds and four others for 1 gigabit.
Other information posted to the forum, Rivers says, indicates that the switches use a central 10-gigabit chip known as Scorpion, a processor manufactured by Broadcom. Broadcom now offers a more advanced 10-gigabit chip known as the Trident.
Google was at the forefront of the move to 10-gigabit Ethernet networks, and the technology is now starting to spread across the industry. Facebook just recently moved to 10-gigabit Ethernet inside its new data center in western North Carolina.
The question is whether the Facebooks and the Amazons will also follow Google’s efforts to significantly cut the cost of its networking gear. But they’re already moving in that direction.
In designing its own gear, Google also reduced costs by removing the middleman. Rather than purchase switches from big-name hardware sellers such as Cisco and HP, it took its custom designs to manufacturers in Asia — the same manufacturers who build hardware for the Ciscos and the HPs. In recent months, other web giants have followed suit, looking to buy their own dirt-cheap gear directly from Asian manufacturers.
They’ve yet to go as far as Google. Odds are, they’re just buying commodity gear from Asia rather than coming to manufacturers with their own designs. But this may change. They too need ways of cutting big costs across their data center networks — and Facebook is already designing all sorts of other hardware for the data center.
The other web giants may not have access to a Pluto Switch. But you can bet they’re looking at something an awful lot like it.
( via wired.com )