Light-linked chips could train AI faster with less energy

Fiber-optic technology helps chips communicate with each other at the speed of light, allowing them to transmit 80 times more information than using traditional electrical connections. This could significantly reduce the training time required for large-scale artificial intelligence models from months to weeks, while also reducing data center energy and emissions costs.

Most cutting-edge computer chips still communicate using electrical signals transmitted over copper wire. But as the tech industry rushes to train AI models at scale, a process that requires networks of AI superchips to transfer large amounts of data, companies are using fiber optic speed-of-light communications to link chips together. I am very passionate about this.

This technology is not new. The Internet already relies on undersea fiber optic cables that stretch thousands of kilometers between continents. But to transmit data between fingernail-sized chips, companies need to connect as many hair-thin optical fibers as possible to the end of each chip.

“As we all know, the best communication technology is fiber optics, which is why fiber optics is used everywhere else for long-distance communications,” Mukesh Khare of IBM Research said at a press conference previewing the technology. said. “This co-packaged optical innovation essentially brings the power of fiber optics to the chip itself.”

Khare and colleagues have developed an optical module that allows chipmakers to add six times more optical fibers to the edge of a chip than with current technology. This module uses a structure called an optical waveguide to connect 51 optical fibers per millimeter. It also prevents optical signals from one fiber from interfering with adjacent fibers.

“What IBM has really done here is use its materials and packaging technology, all of its history of leadership in that field, to really break free from the way high-density fiber optics is done using waveguides. ,” said Dan Hutchison of semiconductor technology company TechInsights. A research company headquartered in Canada. “For me, when I saw this, it was a big step forward.”

The result is enhanced chip-to-chip communication, potentially allowing AI developers to train large language models in less than three weeks instead of three months. Switching from wires to fiber optics for chip communications could also mean cutting energy costs for training such AI models by a factor of five.

IBM has already put its optical modules through stress tests that include high humidity and temperatures ranging from -40°C (-40°F) to 125°C (257°F). Hutcheson expects major semiconductor manufacturers may be interested in licensing the technology.

“We are in the early days of all of this, but semiconductor technology is the hottest area right now in terms of high-performance computing and AI technology,” he says.