Today, we are thrilled to introduce Envise — the world’s first general-purpose photonic artificial intelligence (AI) accelerator. Inference, which is synonymous with the deployment of AI models, represents the fastest-growing, largest energy footprint endeavor in the AI computing space. Envise enables ultra-high performance inference on the world’s most advanced AI models (GPT-3, Megatron) as well as staple neural networks (BERT-Large, DLRM, ResNet-50).

Envise will be offered in a 4-U server blade form factor — enabling customers to tackle the most advanced workloads with an array of 16 Lightmatter Envise chips. Each Envise 4-U server blade offers up to 6.4 Tbps of optical interconnect for multi-blade scale-out, 1TB of DDR4 DRAM, and 3TB of solid-state memory.

To enable our customers to extract maximum performance from Envise, we are introducing our software stack: Idiom. Idiom compiles and executes neural network models on Envise and offers tools for debug, profiling and model optimization. Idiom also enables automatic blade detection — enabling rapid deployment of large AI models across entire server racks.

It’s a momentous day for Lightmatter. For the past three years we’ve been hard at work inventing the foundation upon which this new class of computers operates. It hasn’t been easy, but we’ve kept a steady eye on our motivating principles:

1. Human progress is inextricably linked to progress in computing.
2. Our environment is fragile and likely irreplaceable; reducing emissions associated with the growth in computing is important.
3. The story of computing has been one of democratization; the world is better off when everyone has access to the fruits of technology.

We couldn’t be announcing our products at a better time: migration to next-generation transistor process nodes is slowing and that’s just the tip of the iceberg. Over the past 15 years, process node advancement has delivered marginal reductions in transistor energy consumption — making it increasingly challenging to deliver power to computer chips and to keep them cool. This translates into massive energy consumption in data centers and a rapidly growing, planet scale environmental impact.

We are proud of the work we’ve done on Envise, but we know it isn’t complete without solving the communications bottleneck. Today, we’re announcing Passage—a new, wafer-scale programmable photonic interconnect through which computer chips communicate at unprecedented speed. Passage, an 8" by 8" computer chip, integrates transistors and photonics (including lasers, modulators, and photodetectors). Heterogeneous computer chips are integrated with the platform using standard chip-on-wafer packaging technology. We think Passage will enable a world where configurable-topology supercomputing systems are built on a single platform that takes advantage of the benefits of optics, without the packaging cost and complexity. 100Tbps chip-to-chip interconnect.

Interested in joining our early access program and getting your own Envise blade or partnering with us on the future of interconnect? Visit lightmatter.co.

We’re just getting started.

Introducing Envise, Idiom and Passage — Next Generation AI Compute, Compile and Interconnect… was originally published in Lightmatter on Medium, where people are continuing the conversation by highlighting and responding to this story.

In previous blog posts from Lightmatter, Martin Forsythe [1] discussed how general matrix multiplication (GEMM) is powering most advanced neural networks being developed and applied today. He also compared implementing GEMM processing in pure digital electronics to integrated photonics. Mike Gould’s [2] subsequent blog post described the various optical components that can be combined to realize a GEMM calculation in an integrated photonics device. This was followed by Ozgur Yildirim’s [3] post which described the integrated analog components that are being designed to drive the photonic GEMM implementation. Here, we will cover Design Verification (“DV”) as it pertains to the kind of mixed-signal, mixed-domain circuit design we do at Lightmatter. We discuss our methodology for modeling analog and photonic components in Python, which runs alongside the digital portion of our design.

DV is an essential part of any product design process. At its core, DV involves validating the functionality of a product against target specifications. Because chip design and fabrication is a highly capital intensive endeavor (advanced node tapeouts can cost more than $10,000,000), DV is a critical part of any chip design flow. After decades of refinement, standardized and trusted protocols have been developed for validating digital circuit designs. Validation protocols for chips that contain a mix of digital, analog, and opto-electronic components are less refined. This typically involves applying directed random stimulus to the design under test and checking that the design produces the correct outputs.

DV engineers and digital designers typically write code in Verilog. Analog designers and photonics designers typically favor MATLAB, C, or Python. At Lightmatter, we have agreed on Python as a common exchange language between teams for its ease of use and learnability.

System and component overview

A conceptual drawing of Lightmatter’s optical compute core is shown in Figure 1. The optical core performs a GEMM operation. The system feeds the GEMM with input vectors and weights, and records the corresponding output vectors.

Figure 1. Conceptual drawing of Lightmatter systems, including digital to analog converters (DACs), analog-to-digital converters (ADCs), transimpedance amplifiers (TIAs), and photodetectors (PDs).

As indicated in Figure 1, a large portion of the design is outside of the digital domain, which renders traditional digital techniques insufficient. To simulate this system, we created a methodology for modeling analog, optical, and electro-optical components and their interfaces.

The SystemVerilog language and simulation

In a typical chip design, the system is described in a Register Transfer Level (RTL) [4] language. Over decades of maturation of the chip design process, many RTL languages have been used. Today the industry standard is SystemVerilog, which is an extension of the Verilog language. The SystemVerilog language is a unified hardware design, specification, and verification language. It is codified in an IEEE standard which includes support for behavioral and RTL hardware descriptions as well as test benches that provide stimulus and checking. SystemVerilog also provides application programming interfaces (APIs) to foreign programming languages [5].

SystemVerilog support for both behavioral modeling and APIs for foreign programming languages enables us to extend the standard digital design process to include models of analog and photonic components, which are integral to our design.

Real number modeling

SystemVerilog simulators as described in the IEEE standard are discrete time, event based simulators [6] — they sequentially simulate discrete units of time and only model events or signal changes. For pure digital designs, this leads to efficient simulations of the binary values of signals even for large synchronous designs. Event-based modeling is not sufficient for analog and photonic components since the output quantities of these devices are generally continuous.

SystemVerilog is primarily focused on synchronous, clock-based designs with binary values, but it also has the ability to use real-numbers in behavioral model semantics. In the design industry, this methodology is known as “real-number modeling”. We use real-number modeling to represent analog and photonic properties including voltage, current, light intensity and phase. Below, we consider real-number modeling of DACs as an example.

Figure 2. DAC driving transmission line and photonic load.

At the architectural exploration phase, a simple SystemVerilog function was used to model the DAC’s transfer function. We also used a component called a Finite Impulse Response (FIR) filter [7] to model the combination of the transmission line and the load. Below is a simple code snippet which shows the function which converts the DAC input code to a real-number voltage.

Figure 3. Verilog code implementing Digital to Analog Converter (DAC)

Figure 4. Simulation of ideal DAC and FIR transfer function.

With the code shown in Figure 3, we are able to explore a wide range of architectural possibilities, including how many bits of precision the DAC should use, what voltage range it should cover, and more. The simulation waveform in Figure 4 shows a simple transmission line delay driving a capacitive load. Virtually any transmission line and load characteristics can be simulated through appropriate choice of the FIR model coefficients, enabling rapid exploration of many system configurations.

Real-number models are also applicable later in the design process when design parameters are known and the DAC circuit has been fully designed and characterized. A lengthy Spice circuit simulation [8] of the DAC driving the transmission line and load can be captured and programmed into the FIR model coefficients giving sufficient accuracy in a fast discrete time simulation that can run alongside the simulations of all of the digital logic and memory.

Co-simulation SW architecture

As the design progresses, various analog and photonic component models become more detailed and accurate. We still need to simulate those components along with the digital control logic and memory.

In addition to detailed component models, we have a high level Instruction Set Architecture (ISA) model [9] which is used by the software development team. This model is used as a reference for the complete RTL/analog/photonic system. This insures that the hardware matches the intended behavior and that fast software simulations match the behavior they will see on the real hardware. A wide variety of both high level and detailed models are co-simulated in our standard Verilog design. This prompted us to develop a full-featured, multi-language co-simulation software toolset.

Our initial models spanned a variety of languages including SystemVerilog, C/C++, Python, and combinations of these. Figure 5 shows an example co-simulation software architecture.

Figure 5. Co-simulation SW architecture

The shared library linking the SystemVerilog DPI to the Python interpreter is written in C, but it is mostly auto generated from Python and SystemVerilog code and function prototypes. The models developed are used across various phases of the chip design process including architectural exploration, component-level design, subsystem RTL, and full-system simulations including the software stack and device drivers.

Why Python?

Python is designed to be highly extensible and expressed using a simple and easy-to-read syntax. Python’s standard library is full-featured and provides more than 130,000 packages to extend base functionality. As an example, the Numpy (numerical Python) library implements a wide variety of linear algebra functionality as well as complex scientific functions that are widely used in machine learning research and development.

Modern chip design teams are multidisciplinary with engineers specializing in digital, analog, and now photonic design. Across these disciplines, Python has become the “lingua franca”. Similar to the way English is the common language of pilots and air traffic controllers worldwide, Python is “spoken” fluently across the disciplines of most modern chip design teams.

Python is an interpreted language, so design changes are relatively quick, avoiding the compile/link steps of many other languages. With proper coding conventions, models can be shared between disciplines. Faster iteration on the part of programmers helps to quickly identify and remove undesired interactions between simulated components.

Due to the very rich Python libraries available, much of the analysis and development of the component models can be performed outside of the SystemVerilog simulation environment. This enables cross-functional development and rapid design iterations.

While the compile/link step is not generally considered cumbersome for software-only teams, large chip designs often require 30 minutes for a SystemVerilog simulation to compile. The ability to change a Python component model, or test stimulus, without the need for the 30 minute compile/link step enables increased productivity and reductions in overall chip design schedule.

Multi-language co-simulation implementation decisions

This section describes the major decision points for a custom co-simulation implementation.

SystemVerilog simulator interface

SystemVerilog already has a well defined interface to external functions called the Direct Programming Interface or DPI. The DPI allows bidirectional access to the SystemVerilog simulation. External functions can call SystemVerilog functions and SystemVerilog can call external functions. We chose the latter. SystemVerilog masters the simulation and calls Python functions.

In general for models of analog or opto-electronic system components, SystemVerilog shell models were made. These shell models were connected to other portions of the system and managed by place and route tools in the back end of the physical design process. From these relatively lightweight shell models, external models were accessed via function calls.

Although it will not be covered in detail in this blog post, we provide access to the Python interpreter. This allows production software to be used as stimulus in the simulation environment.

Cross language data considerations

For our system, memory management and cross language data conversion needed to be solved cleanly such that programmers are isolated from the internal cross language complexities.

We needed to handle the following cross language data and memory models:

• SystemVerilog objects are statically typed, dynamically allocated and garbage collected
• Python objects are dynamically typed, dynamically allocated and garbage collected
• C and C++ are statically typed, user allocated as well as de-allocated (malloc, free)

In our case we chose to make SystemVerilog the memory owner for cross language data. Any variables or structures that needed to pass between SystemVerilog and the external models were created in the SystemVerilog domain, and either “passed by value” for simple integers or real-numbers or “passed by reference” in which the external model was simply given a handle to the SystemVerilog structure and SystemVerilog managed it’s lifetime. This also allows C/C++ models to avoid memory management.

Python to C tool suite

Python calling a foreign language is typically called “extending” the language, while a foreign language calling Python functions is called “embedding”. We use embedding.

There are multiple options for embedding support. We chose CFFI for C Foreign Function Interface [10]. The CFFI tool suite is primarily designed to extend Python, but it also has very good support for embedding Python, which is the direction we implemented. To simplify its use, it adheres to the philosophy that most of the embedding logic is written in Python, and the conversion of datatypes for Python objects to/from C is auto generated.

The CFFI build process generates a shared library in which Python functions are exported as C functions. The shared library invokes a single instance of the Python interpreter to execute the Python functions and takes care of any necessary data conversion.

Once the shared library generated above is linked with the SystemVerilog simulator executable, the exported C functions are directly available within the SystemVerilog simulator through the API. These functions then appear to be native SystemVerilog functions to the remainder of the SystemVerilog code.

This entire process is automated within our make based build flow, so the users are mostly isolated from the intermediate implementation. With just SystemVerilog and Python coding knowledge they are ready to start writing the models.

Performance

In any co-sim environment, performance is critical in order to allow sufficiently complex tests to be run within a reasonable time (e.g. hours, not days). Cross language calls can have a significant cost so they should be made only when needed. In our system, Python model calls are only made when the SystemVerilog shell detects a signal change which would affect the component.

With appropriate attention to performance and some code tuning along the way, we achieved the following performance:

• For functions with simple data types (non arrayed integer or real variables), the Python call and return takes 3us per call.
• For functions with single dimension arrays of up to hundreds of array elements, the raw Python call and return time is 50us per call. The extra time is due to the array data conversion: SystemVerilog ← → C ← → Python

This level of performance allows fine grained models to be implemented in Python, even if they end up getting called every clock cycle. A SystemVerilog simulation of a chip composed of billions of transistors is computationally intensive. In our experience, the Python interface calls make up a small percentage of the total simulation time.

Putting it all together

This section provides an example of a simple analog model implemented in Python rather than purely within the SystemVerilog language as was shown in the earlier example.

The example used here is for a current based Analog to Digital Converter (iADC).

Figure 6. Verilog code snippet with imported functions implemented in Python

The code given above gives an example of a SystemVerilog shell for a model in which the implementation details of the model are actually in Python Code.

The initial block executes at the beginning of simulation time and creates a Python object instance to correspond to the SystemVerilog shell instance. The always@* code block executes any time a variable in the right side of any expression changes. The code executed in that case reflects any state changes in the SystemVerilog model, into the Python object instance, so the Python object can now use those new values.

Figure 7. Python code snippet

The Python snippet shows the simple code which implements the Python function definition which is made available to SystemVerilog. The @ffi.def_extern() is a Python decorator defined by the CFFI package. This decorator, combined with the C function prototype, auto-generates all the data conversion code as well as exports the C function in the generated shared object library. When the function is called it creates a Python object instance and stores that instance in the iadc_dict dictionary, thus creating a Python instance for each SystemVerilog instance.

As these examples show, engineers with Python familiarity can now implement complex models in Python. And, more importantly, digital designers can reuse Python models that analog or photonics engineers have already made. The complexities of the framework connecting the two languages is largely hidden.

Conclusions

Lightmatter is implementing a machine learning accelerator containing a mixture of digital, analog, and photonics elements. We are developing a methodology to model many of these elements in Python and link them with detailed SystemVerilog simulations of the system.

We have performed the design verification process on our mixed technology design with excellent results. We gain significant flexibility in the mixed technology modelling and we performed much of the component design process purely in the Python domain.

References

1. Martin Forsythe Matrix processing with nanophotonics
2. Mike Gould Optical components for matrix processing
3. Ozgur Yildirim Analog Interface of Optical Processors
4. Wikipedia: Register Transfer Level
5. IEEE Std 1800™-2012 (Revision of IEEE Std 1800–2009) IEEE Standard for SystemVerilog — Unified Hardware Design, Specification, and Verification Language
6. Wikipedia: Discrete-event simulation
7. Wikipedia Finite impulse response
8. Wikipedia SPICE (“Simulation Program with Integrated Circuit Emphasis”)
9. Wikipedia Instruction Set Architecture
10. CFFI documentation

Techniques for performing design verification of an optical processor was originally published in Lightmatter on Medium, where people are continuing the conversation by highlighting and responding to this story.

The figure below shows a matrix multiplication (a), its block diagram (b), and implementation using optical and analog systems (c). The value of each element in the input vector and weight matrix needs to be converted into the analog domain, which can be implemented by digital-to-analog converters (DAC). The number of bits in each DAC determine the precision of the corresponding vector or matrix element.

Optical core drives an array of photodiodes (PDs). The currents generated by these photodiodes are amplified by an array of transimpedance amplifiers (TIAs) and then converted into the digital domain by an array of analog-to-digital converters (ADCs). The number of bits in each ADC determine the precision of the output vector.

In this blog, we will go over the operation principles and performance parameters of these analog blocks.

High-speed medium-precision DACs

A digital-to-analog converter is an electronic circuit that converts a multi-bit digital signal into an analog signal with multiple levels as shown in the below picture (a). Resolution, maximum operation frequency, settling time, and power consumption are the main performance parameters of a DAC. In a linear binary-weighted DAC, an N-bit digital input corresponds to 2^N equally separated analog levels at the output. 2^N is the dynamic range of this DAC, as it is the ratio of the biggest possible step and the smallest step at the output. Deviation from the ideal transfer function is quantified with integral nonlinearity (INL) and differential nonlinearity (DNL) as shown below (b). If the DNL is smaller than the least significant bit (LSB) step size, the DAC becomes monotonic, which means that with the increasing digital input code, the output can only increase.

In general, a linear, monotonic, high-resolution DAC dissipates a lot of power and occupies a large silicon area. In our artificial intelligence applications, we adopted mid-resolution DACs for optimum performance, as they can run at very high speed, driving the optical core with high bandwidth, while still being power efficient.

Amplification of photodiode current

Photodiodes have high-speed current outputs. These currents are converted into the voltage domain by transimpedance amplifiers (TIAs). For the given power consumption, there is a trade off between dynamic range and bandwidth of the TIA. The TIA needs to have a very high bandwidth to leverage the high performance available in the optical domain. To meet the high bandwidth requirement, the feedback resistor (shown as R in the picture below) cannot be very large. As R is also equal to the gain of the TIA, to achieve rail-to-rail swing at the final output, an additional voltage gain stage is needed after the TIA. This final output signal is sampled and converted into the digital domain by the ADC.

High-speed medium-precision ADCs

An analog-to-digital converter is an electronic circuit that converts analog signals into digital signals as shown in the below picture. First, the analog input is sampled and held on an internal capacitor. Next it is quantized into a digital signal after multiple comparison steps. The real resolution, which is known as ENOB (effective number of bits) including high-speed and nonlinear effects, maximum sampling frequency, and power consumption are the main performance parameters of an ADC.

One of the benchmarks for ADCs is the Walden figure of merit (FOMW) shown in below equation, where P is power consumption, f_S,Nyq is the Nyquist sampling frequency.

The picture below shows the FOMW of all the ADCs published in the two main integrated circuits conferences in the last two decades. This graph suggests that to double the sampling frequency of an ADC on the state-of-the-art envelope, you need to quadruple the power consumption. From the equation we can see that for each additional bit (for example if you increase ENOB from 9 to 10, or from 11 to 12) you need to double the power consumption.

This implies a high-cost design space for high-speed mid-resolution ADCs needed by artificial intelligence applications. However, this state-of-the-art boundary moves down each year, as many of the most efficient ADCs were designed in the last couple of years. The reason for this positive trend is the availability of higher-speed process nodes and new circuit design techniques, such as novel ADC topologies and new amplifier designs used within the ADCs. Here, at Lightmatter, we are applying multiple of these new methods to develop power efficient, high-speed, mid-resolution ADCs that can digitize the high bandwidth outputs from the optical core without adding a significant power overhead to the total power consumption of the chip. This efficiency rule also applies to our DAC and TIA designs.

Analog Interface of Optical Processors was originally published in Lightmatter on Medium, where people are continuing the conversation by highlighting and responding to this story.

In our previous blog posts, we introduced the idea of using silicon photonics to perform large-scale linear arithmetic operations faster and more efficiently than can be done with digital electronics. As you may recall, our systems make use of many identical devices called Mach-Zehnder interferometers (MZIs) [1] to implement a basic linear operation, a 2x2 matrix multiplication, from which much larger matrix multiplications can be built up. In this blog post, we aim to follow up with a more detailed description of how MZIs work, and hopefully elucidate some of the advantages of silicon photonics over digital electronics for performing arithmetic.

Scalar MZI

We first consider an MZI that can be used for scalar multiplication. A basic layout is shown below. A single input waveguide is split symmetrically to become two arms of the interferometer. As the light travels down each arm, the electromagnetic waves pick up a phase, which is (in general) path-dependent, before being recombined into a single output waveguide. We’ll refer to this configuration as a “scalar MZI.”

A scalar MZI can be divided into 3 sub-sections: two Y-junctions and a differential phase-shift. Before we go through the analysis of the full system, let’s first get acquainted with these sub-systems.

Y-Junctions

A Y-junction is a 3-port integrated optical device with a single port (A), coupled equally to a pair of ports (B and C). A simple version can be implemented by connecting a straight waveguide to both left- and right-turning waveguide bends as shown below.

When sending light into port A, the light signal undergoes a transformation that can be described by the following transfer matrix:

In order to obtain the optical field amplitudes in the output waveguide modes (B and C), we can multiply the input field amplitude by the transfer matrix. Assuming the input amplitude to be 1, we obtain identical output field amplitudes:

Much as one might expect, the input light at port A is simply split in two, with half of the optical power in each of the output waveguides (optical power is proportional to the magnitude squared of the field amplitude). In this situation, the light behaves much like water in a system of pipes.

Let us now examine what happens when we input light at port B. By optical reciprocity (a principle in optics that says light can travel both directions down the same path — think about the lenses of your glasses), the transfer matrix for passing backwards through it is simply the transpose of the forward transfer matrix:

Using the same matrix multiplication formalism, we obtain:

Note that only half of the input power made it to the output, and the remainder seems to have magically disappeared (in reality it was radiated out of the system). In this case, the wave nature of light causes it not to behave like a classical fluid (water in a pipe), as can be seen in the simulation below.

Let’s now consider one final situation, wherein light is launched into both port B and port C. In this case, we can arbitrarily set the input field amplitudes to b and c respectively. Once again, we multiply the input amplitude vector by the transfer matrix to obtain:

This final Y-junction result illustrates a fundamental concept in optics known as superposition: a system’s response to multiple excitation sources is simply the sum (or superposition) of the responses to each individual source. This is one of the main advantages of optics for performing linear arithmetic: as a direct result of the inherent linearity of the system, addition is as simple as combining (e.g. interfering) two optical modes. Further, the addition operation is effectively free: no power consumption is required! Note that generation of the input light does require power consumption but this should not be included in the cost of the addition operation, as is discussed toward the end of this post.

Phase Shift

The scalar MZI without a differential phase shift section is a useless device that transmits all input light to the output. In terms of arithmetic operations, it can be thought of as dividing a number into equal halves and subsequently adding those two halves together again. However, as we will see, the introduction of a phase shift between the two arms results in a change in the optical power transmitted to the output mode. The transfer matrix for the phase shift section can be written as:

There are many ways of introducing a phase difference between the two arms, but they are all essentially based on delaying the wave-fronts in one arm versus the other.

Perhaps the simplest method is to make the arms of different length. Because the waves in each arm travel at the same speed, those in the longer arm are delayed with respect to those in the shorter arm, resulting in a relative phase difference:

where n is the refractive index of the waveguide, ΔL is the length difference between the two arms, and λ is the free-space wavelength of the light.

Let’s Put the Pieces Together…

The system’s transfer matrix is obtained by multiplying the transfer matrices of the component parts (confusingly, the transfer matrices are written out in the opposite spatial order of the elements assuming the light travels from left to right). The resultant transfer matrix for the scalar MZI can thus be calculated as:

The output of the scalar MZI, then, is simply its input multiplied by a scalar value (cos(φ)). This illustrates the second key functionality required for linear arithmetic: scalar multiplication. And once again, just like addition, scalar multiplication using optics does not require the dissipation of energy!

Electro-Optic Scalar MZI

Rather than relying on optical path-length, programmable phase shifts typically rely on changing the speed of the waves in the guiding material, i.e. its refractive index. Depending on the material in question, there are several common ways of accomplishing this. These include the thermo-optic effect, in which the refractive index of a material changes with temperature; the plasma dispersion effect, where the quasi-Fermi level of a semiconductor (concentration of electrons and holes) changes its refractive index; and the electro-optic effect, wherein application of an electric field to a material induces a proportional change in its refractive index. Using any one of the above effects enables the implementation of a programmable phase difference.

Let’s now consider a specific example: let the waveguides in the phase-difference section be made of an electro-optic material. In this case, opposite electric fields applied across each arm can be used to induce a differential phase between the arms. These electric fields can be obtained with a single voltage, as illustrated below.

In this case the phase difference can be written as:

where c is the electro-optic coefficient of the material, d is the distance between the electrodes, L is the length of the electro-optic section, and λ is the free-space wavelength of the light. For an input field amplitude A, the output field amplitude of the MZI is given by:

From this example we can see that the scalar multiplier implemented by an MZI can be encoded in a voltage. This is an important point as it allows the value of electro-optic multipliers to be programmed using high-speed electronics, enabling a fully integrated reconfigurable solution. Though changing the multiplier value requires the transport of charges through an electric potential and thus energy dissipation, maintaining a static multiplier value does not, provided in this case that the electro-optic material is non-conductive.

Vector MZI

Having now seen how the “scalar MZI” can be used for scalar multiplication, we will now modify it into a “vector MZI” to perform matrix-vector multiplication. A schematic version is shown below:

Once again, we can divide the system into 3 sub-sections, but this time we replace the y-junctions with 50:50 directional couplers.

Directional Couplers

Directional couplers are 4-port integrated optical devices with two pairs of ports (A/B and C/D). Each port in a pair is coupled to both ports in the other pair, but not to its partner. So if, for example light is input into ports A and B, the output amplitudes at ports C and D will each be linear combinations of the input amplitudes. Such devices are realized by bringing two waveguides into close proximity so that they are evanescently coupled to one another, as shown below:

A version of a lossless directional coupler can be represented by the 2x2 unitary transfer matrix:

2x2 Unitary Operation

As in the case of the scalar MZI, we can obtain the transfer matrix of the system by multiplying the transfer matrices of the sub-components in the correct order:

Note that in this case the resulting matrix is a 2x2, rather than a 1x1 as in the case of the scalar version. Much like its component directional couplers, the output field amplitudes are linear combinations of the input field amplitudes; in fact it is correct to consider the vector MZI to be a programmable directional coupler. The arithmetic operation of the device is identically multiplication of the input amplitude vector by its transfer matrix. As many of those reading will recall from introductory linear algebra courses, matrix multiplication is simply an organized series of scalar multiplications and additions: the two basic operations we have already identified as energy-free in electro-optic circuits. It should come as no surprise that this compound operation can also be performed with zero power consumption.

By stringing vector and scalar MZIs into a suitable network along with simple phase shifters, arbitrary matrix multiplication can be implemented, and again the multiplication can be performed without consuming any power. It should be noted that it is necessary to consume power at the inputs and outputs of the network to encode the input vectors (this includes light generation), and decode the output vectors. However multiplying an N x N matrix by an N x 1 vector requires only O{N} encode/decode operations, while requiring O{N²} scalar multiplications and additions. Digital electronics require the dissipation of energy to perform scalar multiplications and additions. From the dramatically lower scaling of energy-consuming operations required in optics, it should be clear that there is some value α for which when N > α, optics will always be more efficient.

A challenge here at Lightmatter is engineering our systems to both reduce that value of α, and increase the value of N for which we can build practical systems. By reducing the value of α, we will broaden the range of applications for which optical computing is attractive and increase our impact on reducing the energy costs associated with computing. By increasing the attainable value of N, we will do our part to extend the reach of human capability, enabling computation at scales unachievable with conventional digital machines!

Optical components for matrix processing was originally published in Lightmatter on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Lightmatter, we are building a photonic processor to accelerate deep learning. In a previous post we discussed the history of optical chips and the advances that made Lightmatter’s technology possible today. This post will dive into some of the technical differences between optical and electronic accelerators with the goal of explaining why nanophotonics offers a route to higher computational performance while requiring significantly less energy, reducing both cost and heat production. The post will first explain how general matrix multiplication works and how it is implemented in accelerator hardware today. Then, we will explain how Lightmatter’s nanophotonic processor uses light rather than digital electronics to implement the same GEMM calculations more efficiently.

Matrix multiply-accumulate operations

A matrix is a two-dimensional array of values — its size is quantified by the number of rows and columns, which is often written as [rows 𝗑 columns]. When two matrices, A and B, are multiplied together, the result is a new matrix D. If A and B are of sizes [I 𝗑 J] and [J 𝗑 K] respectively, the matrix D is of size [I 𝗑 K]. Each element of D is computed by accumulating the multiplication of each element of the vectors (1D arrays) that are extracted from the rows of A and the columns of B. Consequently, matrix multiplication “contracts” the inner dimension J.

In code this looks like:

When designing hardware to perform matrix multiplication there are a number of optimizations that are possible. In digital electronics, one such optimization is to implement a hardware instruction called a multiply-accumulate (MAC), which computes d←a∗b+c for three numbers, a, b, and c. From the code above we can recognize that a matrix multiplication can be decomposed into a series of MACs. There are also a number of optimizations involving data access/memory management tricks, but for the purposes of this post we’ll just concern ourselves with the MACs since they are the fundamental unit of computation.

If you look back at the first figure, you will notice that the elements in A, B and D are grouped suggestively. It turns out that you can express the multiplication of a large matrix in terms of the products of smaller matrices if you split the larger matrices into sub-matrices, called tiles or blocks.

On a matrix processor, such as Google’s TPU [2] or MIT’s Eyeriss [3], rather than operating on a stream of single values in row and column vectors, the computation is parallelized by processing products of tiles. The tiles are accumulated in exactly the same row-column pattern pictured above where the unit element was a single entry.

So far, we have explained the matrix-multiply portion of the GEMM operation. The word “general” in the acronym comes from allowing the matrix product (A 𝗑 B) to be summed with an initial value matrix C [4], forming a matrix multiply-accumulate (MMAC).

In this equation the left arrow indicates that the result may optionally be accumulated in-place, i.e. into the same memory that was storing C, just like the += notation in the code above. Thinking in terms of MMAC operations we can visualize the matrix multiplication of the tiles as follows:

Matrix processors — mesh-based parallel computing

To understand how nanophotonics and digital electronics differ for making GEMM accelerators, it is helpful to first discuss the mesh diagrams that get used to describe parallel dataflow processors. In a dataflow processor, different processing elements (PEs) independently perform partial computations and data moves between processing elements each tick of the processor’s timing clock. Computation at each PE depends on the upstream elements, i.e. in the diagram below b depends on a, and c depends on b (thus indirectly on a).

Thus, the arrows that connect PEs in a mesh diagram indicate both a temporal and arithmetic dependency structure. Each compute node also plays a role in the distributed storage of data by having a small local memory. However, due to the local topology of the mesh, moving data from main memory to an internal PE requires the data to first pass through intermediate PEs–e.g. it takes two clock cycles to move data from memory to the central PE in the diagram below.

Depending on the dataflow processor in question, each node may have varying levels of complexity; a PE might be as complicated as a general purpose CPU or as simple as a single multiply-accumulate (MAC) unit with data routing. When the mesh of PEs is homogeneous, i.e. each PE performs the same partial computation, the mesh is called a “systolic array” [5].

The pipeline parallelism that is used to feed a systolic matrix multiplier can be a little surprising if you are not used to dataflow meshes. In a spatial accumulation mesh, the values for a tile of A are broadcast to the mesh prior to beginning the computation. Column vectors from the tile of B are then fed into the mesh in a staggered pipeline and for each clock cycle data flows between the processing units and accumulates the matrix-vector product as it passes through. The animation below shows this pipeline for a [3 𝗑 3] systolic array processing a [3 𝗑 9] input matrix.

The animation above shows that computing this [3 𝗑 3][3 𝗑 9] matrix product takes 14 clock cycles. In general it takes N timesteps to load the tile A onto an [N 𝗑 N] mesh and (2*N-1+K) clock cycles to compute a [N 𝗑 N][N 𝗑 K] matrix product. In contrast, a processor with a single MAC unit would take N*N*K clock cycles (e.g. 81).

Weight stationary processing

In deep learning applications, we are frequently interested in computing matrix products where one of the dimensions is significantly larger than the other. For instance, computing a 2D convolution as a matrix product using the so-called im2col trick will result in a small “weight” matrix A and a wide “data” matrix B. In such a situation it is useful to perform as much computation as possible while the same weight matrix tile is loaded on the matrix processor, i.e. keeping the “weights stationary” while streaming “data vectors” through the processor. This weight stationary scheme is typically used in spatial accumulation architectures, while an “output-stationary” scheme is used in temporal accumulation architectures (see references [3, 7, 8]). In the context of deep learning this pattern of holding the weights static is especially useful for “inference” workloads where a pre-trained neural network is used to make predictions on millions of data.

Nanophotonic matrix processors

So far, we have discussed matrix multiply accumulate operations and how they can be implemented with conventional digital electronics. Now we are going to dive into how Lightmatter’s processor uses light instead of electronics. The word “nanophotonic” — “nano” meaning small and “photonic” meaning having to do with light — is used to describe optical computing elements that are implemented with integrated optical circuits, which can be fabricated using traditional semiconductor/oxide processes.

Computing a matrix product on a nanophotonic matrix processor is quite a bit different from the operation of a digital electronics-based 2D systolic array. However, the pattern of using a weight stationary scheme to compute a stream of products between “weight” tiles of the A matrix and “data” vectors from the B matrix remains the same. In fact, we can represent both a systolic array and a nanophotonic processor with the following dataflow diagram in which the processing element (PE) computes the product between an [N 𝗑 N] matrix tile and an N-element data vector. The only difference is that with a systolic array, the data pipelines are staggered across N cycles while a nanophotonic processor computes all N-elements synchronously.

In digital electronics, data is encoded as a binary string in the on/off state of transistors. This means that the numbers that are representable are discrete, e.g. integers or floating-point values. In photonics, data is encoded by modulating the amplitude (or phase) of pulses of laser light; this allows representing continuous values, i.e. real numbers. Changing the strength (or direction) of the optical field changes the real number that is represented. Instead of wires guiding electronic currents, silicon structures called waveguides carry the laser light — this is similar to how an optical fiber guides telecommunications signals. In place of the integrated transistor circuits that perform a MAC, in nanophotonics there are two primitive computational elements for manipulating our representation— a Mach-Zehnder interferometer, that rotates vectors, and an attenuator, that scales them.

The Mach-Zehnder interferometer (MZI) is the most important element to understand because it can be modified to build an attenuator. An MZI operates on the light in two waveguides. By interfering the light from the two “arms” of the interferometer, an MZI computes the product between a [2 𝗑 2] unitary matrix, U, and the 2-element data vector, x, represented by the light in the two incoming waveguides. This product Ux rotates the x vector without changing its length. You can think about this as shuffling some of the light from one waveguide to another while still preserving the total amount of light. The rotation angle, θ, determines how much light moves from one arm to another and is controlled by the voltage applied to two adjustable phase shifters.

There are many ways to parameterize a unitary matrix (see reference [6]), and correspondingly there are a number of different physical layouts for building an MZI. The parameterization used above highlights the relationship with the rotation matrices that you may have encountered in a math class.

The second primitive element for computation nanophotonics is an attenuator. One way to implement this is by blocking off the waveguides coming into/out of one arm of an MZI. Taking the light out of one arm allows this attenuator element to scale the data value in a waveguide by a factor of cos θ, i.e. by factors in the range [-1, 1].

So, how do you use MZIs and attenuators to perform the matrix multiplication between a square tile of A and the corresponding tile of B? Researchers in quantum computation / optics have devised a number of different schemes for making meshes of MZIs that can be programmed to perform any [N 𝗑 N] unitary matrix, U, on N waveguides (see references [9, 10, 11]). You can think of this as building up an N-dimensional rotation by performing a series of [2 𝗑 2] rotations in different pairs of directions, i.e. 2D subspaces. If you view each column of B as specifying the coordinates of a point in a high dimensional space, the act of multiplying this vector by the matrix A can be regarded as transforming the coordinate system.

By using a factorization called the singular value decomposition, it is possible to interpret any matrix multiplication as first performing a rotation, then scaling each axis, and finally performing another rotation. Since a unitary matrix performs a rotation and a diagonal matrix scales each row independently, this decomposition allows us to rewrite any matrix tile, M, as a product of two unitary matrices (U, V) and a diagonal matrix (Σ).

By choosing an appropriate scaling for our number representation, we can implement the diagonal matrix Σ with attenuators. Thus, any matrix tile M can be implemented with two MZI meshes and a column of attenuators. In the diagram below we assume that light propagates from left to right, so the order of the MZI meshes corresponding to U and V is reversed relative to the equation above.

It’s important to note that the lines and boxes in the diagram above have a very different meaning than in the previous dataflow processor diagrams; the entire computation is performed as the light travels from the input to the output of the MZI/attenuator mesh. Since this “time-of-flight” is only about 100 picoseconds, we are able to compute the full output for a matrix-vector product in a single clock cycle!

The future of AI hardware accelerators

ML is both broad enough to apply to many domains and narrow enough to benefit from domain-specific architectures, such as Google’s Tensor Processing Unit (TPU). Moreover, the growth in demand for ML computing exceeds Moore’s law at its peak, just as it is fading. Hence, ML experts and computer architects must work together to design the computing systems required to deliver on the potential of ML.
 — Jeff Dean, David Patterson, Cliff Young IEEE Micro, 38 (2), 21 (2018)

Deep learning is revolutionizing the way that problems are solved in many fields, from enhancing diagnostic medical imaging to powering speech recognition systems and search engines. At the same time, the need for ever more powerful computers to train the largest AI systems has been increasing exponentially —more than 5x faster than Moore’s law! This exceptional computational resource requirement is one of the factors that limits advances in artificial intelligence; all this computation demands considerable energy with correspondingly high financial and environmental costs [12].

The computational throughput achievable with digital electronics is limited not just by how many transistors can be put in a given area of silicon, but by how quickly the chip can be cooled. One of the largest operational costs of a data center is the electricity used to cool the processors. With modern transistor densities, it is not physically possible to cool a chip fast enough to prevent it from melting if all of the transistors on the chip are simultaneously powered on (leading to so-called dark silicon). This also limits how fast the processor’s clock frequency can be (reference [13] offers an approachable discussion). Thus, thermal management is one of the central factors that limits how much computation a chip can perform.

Unlike switching electronic transistors on and off, passing light through the optical elements in a nanophotonic processor does not generate heat. Transistors are still required for the memory used to store data as well as the control electronics that manage the processor. However, the key is that we are able to replace all of the transistors for N*N MACs with just 2N elements for converting the data vector into/out of the optical signal. (There are also electronics for re-programming the phase shifters whenever the weight matrix tile is changed, but these modulate less frequently and are thus less of a power constraint). As a result, our processor uses orders of magnitude less energy per matrix calculation. Since a nanophotonic matrix processor does not generate nearly as much heat, it is theoretically possible to run the processor much faster than the latest electronic chips. By building a processor that is both faster and more energy-efficient than electronic accelerator hardware, we hope to make a positive contribution to the advancement of artificial intelligence and reduce the carbon footprint in the process!

This article contains contributions by Martin Forsythe, Siddharth Trehan, Victoria Pisini, Michael Gould, Tomo Lazovich, Tyler Kenney, Scott McKenzie, Darius Bunandar, and Nicholas Harris.

References, footnotes, and further reading

[1] P. Warden Why GEMM is at the heart of deep learning?
[2] Google’s TPU: arXiv:1709.03395
[3] MIT’s Eyeriss project: eyeriss.mit.edu & arXiv:1807.07928
[4] Strictly speaking the BLAS standard for GEMM also allows scaling by numbers ⍺ and β, D←⍺(A𝗑B)+βC, and optionally allows transposing or conjugating the matrices A and B, but deep learning accelerators are typically designed to perform a simple MMAC.
[5] H.T. Kung (1982) doi:10.1109/MM.2018.112130030
[6] H. Führ and Z. Rzeszotnik (2018) doi:10.1016/j.laa.2018.02.017
[7] C.-P. Lu (2017) arXiv:1705.05983
[8] C.-P. Lu Should we all embrace systolic arrays?
[9] M. Reck et al. (1994) doi:10.1103/PhysRevLett.73.58
[10] W. Clements et al. (2016) arXiv:1603.08788
[11] H. de Guise et al. (2018) doi:10.1103/PhysRevA.97.022328
[12] E. Strubbell et al. (2019) arXiv:1906.02243
[13] P.P. Mattsson Why Haven’t CPU Clock Speeds Increased in the Last Few Years?

Matrix processing with nanophotonics was originally published in Lightmatter on Medium, where people are continuing the conversation by highlighting and responding to this story.

A brief history of optical computing

In the 1980s, scientists at Bell Labs made early attempts at creating optical computers. This new kind of computer would offer bandwidth that was significantly higher than that of electronics — hundreds of terahertz (10^14) compared to a few gigahertz (10^9). By the mid 1980s, hopes for this technology had reached a fever-pitch:

By the mid-1990’s we’ll have flexible programmable computers. You may never know there are optics in there. You’ll see no flashing lights. It will be very dull looking. But it will run circles around everything else. Electronics just can’t keep up with us.

- Dr. Henry J. Caulfield (New York Times, 1985)

The Bell Labs approach to optical computing relied on implementing an optical version of the electronic transistor — a device that is used to switch (or amplify) electrical signals. Unlike electrons, which are used in the transistors inside your phone and computer, light beams do not interact directly with each other. However, light can interact with materials; by temporarily changing the properties of the material that it is passing through, the passage of one light beam can be ‘felt’ by another.

As you may have noticed, the prediction made by the Bell Labs scientists did not come to fruition. This was largely due to difficulties in implementing “optical transistors.” Each optical transistor absorbs some light, making the signal weaker and weaker as it propagates and limiting the number of operations that can be performed on this kind of system. On top of this, there was the issue of storing data optically, a problem which remains extremely challenging to this day. There was a significant backlash in the scientific community towards research on optical computing due to the unfulfilled promises of the 1980s and the hype that it generated.

While the optical transistor was dying, a new kind of optical computing approach was being invented. In the mid-1990s, the field of quantum computing was growing rapidly owing to new proofs that showed that quantum systems could solve problems that were intractable on classical computers. There were many known approaches to implementing quantum systems, including using photons (single particles of light). In 1994, in hopes of building an optical quantum processor, Michael Reck and co-authors described a system that used arrays of a fundamental optical component — the Mach-Zehnder Interferometer (MZI) — to perform an important mathematical operation called matrix multiplication.

At the time, experiments in optics were typically performed using bulky optical components that were screwed into an optical tabletop (i.e., a heavy, metal, vibration-controlled surface; often called a “breadboard”) for mechanical stability. With this “breadboarding” platform, stabilizing optical systems with tens of optical beams that operate in synchronous beats was intractable — even small vibrations or temperature changes would introduce errors to the system. So, while the idea of concatenating small optical circuits to form a larger optical computer was revolutionary, the technology to make it possible had not yet caught up with the theory.

Research on integrated photonics for matrix processing

The solution on the horizon was to shrink the meter-sized optical components down to 10s of microns using computer chips with “integrated photonics” elements that could be easily fabricated and controlled. Thanks to interest in the technology surrounding the fiber optical networks that serve as the backbone of today’s internet, the telecommunications industry was actively working on developing photonic chips. However, it wasn’t until around 2004 that fabricating photonic integrated circuits with a large number of components became feasible. By 2012, photonic chip fabrication facilities began offering multi-project wafer (MPW) services for silicon-based optical chips. This enabled multiple academic research groups to share resources and produce designs in low quantities at reduced costs. The first optical computers that would lead to Lightmatter were created in this environment.

In 2012, Nicholas Harris (Lightmatter’s CEO) and collaborators used the OpSIS MPW service to realize a “programmable nanophotonic processor” (PNP), an optical processor implemented in silicon photonics that performs matrix transformations on light. There were two major technical challenges towards implementing large-scale PNPs: (1) compact, low-loss, and efficient phase shifters and (2) multi-channel control and readout circuitry. The first tapeout was completed at the end of 2012 and the resulting chip is shown above.

As a member of Prof. Dirk Englund’s Quantum Photonics Laboratory at the Massachusetts Institute of Technology, Harris spent the next several years designing the control and readout hardware from scratch, coding the software platform, and developing calibration techniques for PNPs as well as designing and taping-out increasingly complex PNPs. The first research articles on the PNP and its application to quantum information processing were posted to the online preprint server arXiv in 2015: one article from Harris and Darius Bunandar (Lightmatter’s Chief Scientist) together with collaborators at MIT, and another from Jacques Carolan and collaborators at the University of Bristol in the UK. The articles were ultimately published in the academic journals Nature Photonics and Science.

Because the PNP could implement general matrix operations, it enabled a broad array of applications, including classical computing, quantum computing, data routing, security, and more. Installations of the PNP setup were deployed at universities and research facilities including MIT, the University of Vienna, and the Air Force Research Laboratory. We’ve included a list of academic publications that used the PNP at the end of this article. We recently wrote a review article on the field of PNPs, which can be found here.

In 2017, Harris and other collaborators at MIT published a paper on deep learning that used the PNP to demonstrate parts of an all-optical neural network (a concept first reported in 1987 at CalTech). In that work, an entire deep neural network would be unrolled into many PNPs stitched together using nonlinear optical elements (rudimentary versions of optical transistors). As a side note, we’re often asked if this work is a basis of Lightmatter’s tech. It is not — the proposal was interesting, but it had limitations.

First, neural networks can have hundreds of millions of parameters; current technology cannot support a corresponding number of MZIs. In fact, the implementation would likely require arrays of chips optically connected with nonlinear optical elements. Second, there are challenges towards implementing these nonlinear optical elements — the same kind of challenges that Bell Labs had in the 1980s. While implementations of all-optical neural networks from the 2017 proposal are likely to remain elusive, the underlying hardware — the PNP — developed by Harris and collaborators in the Quantum Photonics Laboratory at MIT has established itself as a valuable alternative platform for matrix processing.

Matrix processors and deep learning

While optical matrix processing technology was maturing, big things were happening in the artificial intelligence processor space. In 2017, Google published its work on a matrix processor for deep learning. They called it the “Tensor Processing Unit.” This move was surprising, since Google is broadly known as a software company, yet its team had designed and fabricated a state-of-the-art chip in one year to help enhance Google’s software products. State-of-the-art artificial intelligence algorithms (typically deep neural networks) are, at their core, sequences of matrix products. Therefore, building matrix processors for this task makes a lot of sense.

Google’s TPU release was part of a sort of big bang for machine learning chips as investors, corporations, and entrepreneurs perceived the commercial opportunity in building application-specific chips to speed up artificial intelligence tasks. The realization would spawn internal research efforts at many tech companies and a new wave of hardware startups all aimed at building chips to accelerate artificial intelligence applications.

The TPU is composed of a 2D “systolic” array of multiply-accumulate units (MACs) that fire in a synchronized sequence. Each MAC performs the operation ab+c, where a, b, and c are values stored in registers. A schematic representation of a systolic MAC array is shown above. In Google’s TPU, the MAC array is typically configured so that the weights that represent a layer (or chunk of a layer) of a neural network are loaded into the array and the vector data (images, etc.) flow through the array with each clock cycle. When computing a matrix-matrix product, there are many different data vectors that need to be multiplied by the same weight values. So, there is a big benefit to storing the weights in the MAC array, rather than having to load them repeatedly from memory. This is known as a “weight stationary” scheme and is generally viewed as an efficient method in the context of deep learning inference tasks, since memory access is both slow and energy intensive — especially if the memory is not located on-chip.

The future

At Lightmatter, we take a different approach to processing deep neural networks, but with many similarities to the 2D MAC array employed by the TPU. Our approach, based on the PNP architecture, relies on a 2-D array of Mach-Zehnder interferometers (MZIs) fabricated in a silicon photonics process. To implement an N by N matrix product, our approach requires N² MZIs — the same number of compute elements used by systolic MAC arrays. Mathematically, each MZI performs a 2-by-2 matrix-vector product. Together, the whole mesh of MZIs multiplies an N-by-N matrix by an N-element vector. Computation occurs as light travels from the input to the output of the MZI array within the time of flight for the optical signals of about 100 picoseconds — less than a single clock cycle of your computer! Because our system operates at optical wavelengths, the theoretical bandwidth of the MZIs is nearly 200 terahertz (it would be a challenge to drive the MZIs at anywhere near this speed) compared to a couple of gigahertz for electronic MACs. Our MZIs require orders of magnitude less energy per calculation than electronic MACs implemented in the latest electronic chips.

Because of these special properties of our photonic system, we’re able to implement a matrix processor that is both faster and more energy-efficient than electronic artificial intelligence accelerators.

Since the founding of Lightmatter, we have been improving the way that we control and interface with our optical computer. To make our system user-friendly we’ve undertaken a number of engineering challenges: (1) implementing all of the control and readout circuits on-chip, (2) integrating high density memory in the package, (3) adding standard electronic communications interfaces to the system so that the chip can talk with the outside world just like an electronic accelerator, and (4) building the software infrastructure that will make the system plug-and-play for anyone who will use our product.

Building the hardware and software stack is important, but it can’t be done open-loop. Over the past year, we’ve been benchmarking our system on large-scale, real world neural network architectures and datasets (like ResNet with ImageNet) that are more relevant than handwritten number recognition tasks from the 1990s (like MNIST).

We look forward to sharing our ultra-fast, efficient optical compute platform with companies that, like us, are looking to push the boundaries of what is possible with responsible artificial intelligence.

This article contains contributions by Nicholas Harris, Darius Bunandar, Thomas Graham, Carl Ramey, Martin Forsythe, Tomo Lazovich, Michael Gould, and Victoria Pisini

The story behind Lightmatter’s tech was originally published in Lightmatter on Medium, where people are continuing the conversation by highlighting and responding to this story.

This January marks one year since we started— a lot has happened. We carefully assembled a team of 23 world-class scientists and engineers to develop a scalable platform for high throughput, high efficiency artificial intelligence computing. We taped out our first (successful) test chip with transistors and photonic elements from start to finish in four months. Eight months after that, we taped out a chip with over a billion transistors — a testament to the talent and drive of the Lightmatter team. Looking forward, we will need even more talent and drive to deliver the first photonics-based artificial intelligence accelerator product.

While integrated photonic systems are typically considered in the context of communication in datacenters, we’re leveraging them to address the growing demand for compute, largely driven by artificial intelligence, in the post-Moore’s Law era. It’s worth noting that the end of Moore’s Law isn’t (yet) due to the inability of chip makers to shrink transistors. If you’re going to pack more transistors onto the same sized chip, which has been happening for decades, those transistors need to be commensurately more energy efficient; herein lies the problem. At Lightmatter, we are addressing this scaling challenge in the context of artificial intelligence.

The leading artificial intelligence algorithms (deep neural networks) of today are mainly composed of a single operation: the matrix-vector (or tensor-tensor) product. From a mathematical standpoint, this is perhaps unsurprising since deep neural networks are inspired by networks of neurons in the brain. The connectivity of these neurons, wherein information is believed to be encoded, can be straightforwardly tabulated in the form of an adjacency matrix. Assuming artificial intelligence algorithms continue to take hints from biology, matrix math is likely to be relevant for a long time to come.

Today, digital electronic computers perform matrix-vector products using multiply-accumulate units (sometimes in 2-D arrays). At Lightmatter, we replace this device with an optical counterpart: the programmable Mach-Zehnder interferometer. This photonic device is not bound by the physics that limit transistor-based electronic circuits — opening an avenue towards continuing the currently broken trend of exponential growth in compute per unit area within a practical power envelope.

There’s a lot of work ahead and we’re thrilled to bring the GV team on board as we continue to build out our team and realize a new kind of computer that’s fast and efficient enough to drive the next generation of artificial intelligence.

To learn more about Lightmatter, please visit lightmatter.co.

Lightmatter + GV was originally published in Lightmatter on Medium, where people are continuing the conversation by highlighting and responding to this story.