Connecting Chiplets Isn’t Enough - Solving the Data Movement Challenge in Multi-Die Systems - ARTERIS INC

Good morning, good afternoon, or good evening, depending upon where you are in the world, and welcome to today's 30-minute BitCast, Connecting Chiplets Isn't Enough, Solving the Data Movement Challenge in Multi-Die Systems, brought to you by EE Times, Arteris, and broadcast by AspenCore. I'm Colleen Heckman, and I'll be your moderator today. I have just a few brief announcements before we begin. First the slides will advance automatically throughout the event. You may also download a copy of the slides by clicking on today's slide deck in your resource folder located to the right of the presentation window. Today's presentation will be followed by a Q&A session. You can participate in that Q& A session by asking questions at any time during this webinar. Just type your question into the Q& a text area and hit the submit button. If you experience any technical problems during the webinar just type your issue into that same Q&A text area and hit the submit button we'll be glad to offer you one-on-one assistance and now on to the presentation connecting chiplets isn't enough solving the data movement challenge in multi die systems discussing today's topic is Rick by director of product management and marketing at our terrace Rick has extensive SOC's and embedded software experience with prior design engineering and marketing roles at TI, Broadcom, Silicon Labs, NXP, and foundries.io, Qualcomm. He has a BSc in electronics from the University of Manchester and an Executive MBA from the university of Texas at Austin. It's a great pleasure I now turn things over to Rick to begin. Rick take it away. Thanks Colleen. It's a pleasure to be here with you today. I'm going to talk about six things in my presentation. First of all, what's driving the move to multi-die? Secondly, what changes when the interconnect crosses a chiplet boundary? Then I'll get into why connectivity alone is not enough in multi-die products. We'll talk about the benefits of extending the NOC across chiplet boundaries. I'll touch on the difference between coherent and non-coherent architectures, and then what architects should consider when partitioning a design for multi-dye. So let's look at what's driving the move to multi-dy. Let's look why we're seeing a shift from monolithic to multi-die SOCs. There are a number of reasons here, some of which we'll discuss. First of all, multi-dye allows you to break the reticle limit, the 26 by 33 millimeter maximum dye size you can achieve for a monolithic dye. That's 858 square millimeters. So for example, in the picture on the right, instead of having one very large dye with an array of six by six processes you can break that up into four much smaller chiplets each with three by three processing elements giving you the same number of processes but now where the monolithic die might not have fit within the maximum die size and now the you can build the product that you want with the six by six array of processor elements by using chiplets. And not only that, because you've got smaller die now, you're going to improve your yields because you're gonna be less susceptible to process defects. Multi-die also enables you to use different processes for different functions. So in the example on the right, every die is the same. But in many multi-chip systems, there are a multitude of different designs doing different functions and you might want to build those using different process nodes optimized for their application so for example the compute die could be built or should be built in the leading edge process nodes to get the greatest density and the highest performance but things like io and sram for caches don't benefit from the very latest process nodes. So they're probably fine being built in three, four, or even five nanometer older process nodes and may be more cost effective that way. And then, of course, certain functions like analog and DRAM need their own specific processes. And now it's much easier to integrate those into your product if they are on a separate chiplet that is fabricated in a purpose-built process. And finally, the last benefit we'll talk about here is reuse. So you can build chiplets and configure them into multiple different products to build up a portfolio. Some products in the portfolio will have more of the chiplets. Some products from the portfolio maybe will use different chiplets, But the same chiplet can be used in a large number of products, potentially making it much easier to build out your product portfolio. So here we show on the left how the reticle after shrink gives you a maximum monolithic reticle limited dye size of 858 square milliliters. As I said previously, moving to multi-die, we're taking advantage of silicon interposers to build up 2.5D products with multiple chiplets, and the maximum size of those interposures is growing significantly over the years, as you can see. let's look at the different way these multi-die products are built up out of chiplets uh there are a couple of different ways you can look at this uh first of all who's designing the chiplets in the product is it one single organization uh or are there multiple uh so most of the multi-dye products we're seeing today are single vendor designs where one company is designing every chiplet in the system, the CPU, the GPU, the memory, IO, chiplets, etc. We're beginning to see some companies working together in a partnership to build multi-die products, closely collaborating. One company works on one chiplet, another company works on another chiplet and then they put them together. That's kind of the partner ecosystem model. But I think where many people see the multi-die industry going is an open ecosystem of off-the-shelf chiplets that people can buy and use to build multi-dye products, similar to how you might be buying IP today. Instead, silicon chiplets will become the new IP in the future. So the single vendor use case is a homogenous design. The heterogeneous designs are where you've got multiple teams, multiple designers, multiple companies built in the partner ecosystem or even an open ecosystem building the multi-die product. And those products might need coherent interconnects if you're scaling up cached CPU systems. Otherwise, they're certainly going to need non-coherent interconnects for bulk data movement. So to summarize why multi-die, there are a number of challenges with monolithic designs. reticle limits hamper scalability. The larger chips have lower yield and therefore higher costs. It's harder to reuse or modify IP quickly and doesn't really give you the ability to specialize process technology depending on what circuits you're building. Whereas with chiplets, You can scale beyond the theoretical limits to address your compute demands. Also, by building smaller dies instead of one very large die, you're improving yield and lowering product cost. You can do chiplet reuse to build a product portfolio, and you can optimize the process for the function that you're designing. The multi-die chiplet-based products are real today they're out there you can buy them and they're in the end products that you and I are buying and using you can see products that have two or three different chiplet designs some of them used with multiple instances some designs just have a single chiplet repeated multiple times such as the apple m ultra designs and some uh chiplets are used to build a portfolio of products for example on the left uh the most basic amd epic 9005 is an eight core device that has one of two chiplet designs a compute die and an io die and then at the other end of the scale at the top end of the portfolio there's 128 core version of that epic 9005 product that has a 16 compute die with the same single io die so let's look at what changes when the interconnect crosses a chiplet boundary ideally as little as possible but potentially a lot if you make the wrong choices It's done as in. So when you're designing a monolithic SOC, you've got a network on chip to connect everything together. In this simplified diagram, we show on the left an AXI initiator, perhaps some kind of XPU, GPU or something. And that's accessing an AXi target. We've got on the right-hand side, that could be on-chip cache memory. or memory controller etc so it's sending AXI requests they go to a network interface unit that connects to the network on chip or NOC fabric of the die that requests passes through the NOC fabric and finally makes it to an NIU to its destination are connected to the target that's being addressed in this case the memory there the request is service and a response is sent back maybe if it's a read request then the read data comes back from the memory sent back over the fabric to the initiator so over that not fabric you're not only sending the transactions the axi transactions but typically there's other information being sent through the same fabric but not transaction information things like quality of service data to ensure that priority packets make their way through the fabric and don't get delayed by lower priority data. You might have signals to control different clock and power domains to power things up and shut them down. You might add interrupt data as well. It could be more. So if you divide it into chiplets, then you could put all the initiators perhaps on one chiplet, chiplet a on the left and all the targets on another chip of chiplet b on the right how are you going to connect them together you've got your knock fabrics but uh how's that going to work oh of course there's this thing called a phi and one example of a phi standard is a ucie or universal chiplet interconnect express but how do we connect to those ucie files between the knock and the PHY. The PHYs use interfaces like FDI and CXS. If Unlock doesn't support those, what can we do? Oh, it's okay. There are a number of vendors that not only offer UCIE PHYS, but they offer UCI PHY with AXI to UCI bridges. So now we can put another network interface unit or NIU, between the NOC fabric and the AXI to UCI bridge. And we're good, right? We do that on both chiplets and problem solved. Well, except that the AXi to UCi bridge allows the transport of the Axi transactions, of course. But it doesn't necessarily make it easy for you to transport all the other data, QS, clock, power, interrupts, et cetera. So that non-AXI data is lost unless you go to the trouble to send it out of band to turn that information into AXI packets that can then cross over the bridge. So this is why die-to-die connectivity alone is not. A better way to do this might be instead of converting to AXI, going over the PHY bridge between the chiplets and then converting back to AXi using that information. What if you could transport the native NOC protocol with all of that other information directly over the phy? If you had a NOC die-to-die controller that takes the NOC packets, converts them into a packet format that's compatible with your PHY, then you can send it over that link and then reconstitute it on the other side, and it's almost as if you have a monolithic NOC connecting your initiators and your targets on chiplet A and chiplet B. Just like this. So yeah, that's the way to enable you to do your design as if it was monolithic and then later partition it. So let's go back and look at the AXIT-UCIE bridges. One of the things we've discovered is that some of them do not support bidirectional transactions. So let's consider a different design where both chiplets have initiators and targets. So chiplet A's initiators on the left want to talk to the target of chiplet B on the right. And similarly, chiplet D has initiator on the right that wants to talk the AXI target of the chiplet a on the left. Can we do that? Unfortunately, we cannot, because while the UCI-AXI bridge IP supports requests in one direction and responses in the opposite direction, it doesn't support requests in both directions or responses in both direction. So what you need to do there is you need to add a second AXI to UCIE bridge with a second UCIE PHY on both of your chiplets. So now you've got dual PHY channels, and that adds to the area of your chiplets. It adds tothe power consumption of your ciplets. So maybe not ideal for this use case. But if you've gotten knock tunneling, it doesn't matter. You've just spread the NOC now effectively, logically, across the chiplet bridge using the PHY. So that has no problem using a single PHY channel to support initiators and targets on both sides of the chiplet bridge, enabling AXI transactions to go in both directions, which you couldn't do with an AXII to UCIE bridge. Let's look at the benefits of extending the NOC across chiplet boundaries. I think the biggest benefit is that you can design the interconnect architecture first, before you even think about the physical partitioning. So NOC tunneling, as we're proposing here, makes this possible. So you can do your design as if it is a monolithic product, connect everything up with your NOC fabric all of the transaction data of course plus all of the other out-of-band data all gets transported between all of your targets and all of your initiators and then only then do you need to physically partition the design into chiplets and then you go in and you design the physical interconnect topology but NOC tunneling allows to preserve all the non-transaction data and enabling bidirectional transactions both of which you would lose if you use an axi to ucie bridge so let's look at coherent versus non-coherent architectures so coherency is something that you need are typically in multi-level CPU designs you've got a multitude of CPU clusters and each CPU has its own level one cache a number of CPUs will share level two caches and then a number of CPU classes may then share a level three or system level cache and the data must be kept consistency between all of the caches and the higher levels the level one and level two caches in the CPU clusters are all right back and they may not immediately update the lower level of caches on memory when they get updated by the local CPUs so a cache-coherent interconnect will mitigate that by keeping track of all of the data and knowing which data is the latest and which data has been updated in a high-level cache, and it keeps track of everything for you. So here at Arteris, we provide network-on-chip IP products. We have our ENCOR IP product for coherent NOC designs, and then we have our FlexGen IP product for non-coherent NOC design. And with both of those, we're now engaging early adopter customers building multi-die products out of chiplets. And both of these NOC IP products use the NOC tunneling capability that I discussed earlier, preserving all of that out-of-band knock information like QS, et cetera. The FlexGen multi-die knock-to-knock non-coherent interconnects uses knock tunneling to most efficiently transmit all the data over the chiplet boundary using standard five it's fire agnostic with a highly configurable flit that supports a wide range of industry standards so not just ucie but you could use bunch of wires etc all of the built-in flexgen knock qs management capability gets transported over the die to die link we also have support for up to eight independent receive and transmit virtual channels so another way to prioritize traffic as it goes over the link between chiplets. Similarly, with NCore, we allow our customers to connect multiple chiplets together and maintain coherency across those chiplet links. So what should architects consider before petitioning a design? Well, you need to consider things like the process nodes that are optimum for the different IP that you're using in your design. You need to consider how you're gonna package the design, particularly if you're going to a multi-die design, a 2.5D or even perhaps a 3D. And you also almost certainly need to consider thermal issues. You really need to think about what kind of traffic needs to cross between the chiplets, whether you're doing coherent transactions, non-coherent transactions, or both. And is there other non-transactional data that does need to go from chiplet to chiplet? Things like clock and power domain, control signals, QS, of course, et cetera. And maybe the biggest consideration is who is designing the other chiplets in your multi-die system? Is it you or your colleagues within the same company? Is this a heterogeneous, sorry, homogeneous single-vendor product? Or is it someone from another organization? We argue there's no reason to use a standard protocol for die-to-die chiplet communication unless you're integrating with chiplets designed by a third party. Otherwise, knock-to knock, knock tunneling is simpler and more efficient. So that's why we believe that our Terrace knock-to-knock multi-die interconnect accelerates time to market. Our multi-dye solutions use direct knock communication across the chiplets. We remove that protocol translation overhead, minimizing latency, and most importantly, communicating all of the knock data and sideband signals, including QS, clock and power, domain control, et cetera, et zebra. FlexChain multi-die enables efficient low latency non-coherent data movement, preserves virtual channels and QoS policies, while our N-core fully coherent multi-dye NOC products can support up to four chiplets with up to 4 UCIE links per chiplet to connect coherently all of the CPUs across all of all of the four die. And we plan to address multi-vendor chiplet interoperability in the future using the AMBER C2C standard, but that's a conversation for another day. So that's all I have today. Thank you very much. I'll hand that over to Colleen for questions. Well, thank you so much, Rick, for a great presentation to our audience. We are now moving on to the question and answer portion of the event. If you haven't done so already, please go ahead and enter your questions in now. Please note that we do only have a few minutes left for questions, but we'll get to as many as we can. And if we don't get to your individual questions, someone will get back to you. So Rick, I'm going to jump right in because, again, we've only got a few minutes here. This person asks, can you get me any charts or engineering data for these chiplets? Yeah, thanks for your question. So we're not promoting chiplets ourselves. The companies who are designing monolithic SOCs are now moving towards breaking those monolithic designs those single chip designs into multi-chip designs for the reasons i tried to articulate during my presentation so there are a lot of papers out there that have been published on the benefits of chiplets and different engineering considerations you should make when designing shepherds beyond the considerations i discussed about the interconnect Excellent. Well, thank you, Rick. Okay, moving on here quickly to another question. Once a system crosses a DAI boundary, what changes about data movement, traffic management, and quality of service? Well, ideally, when you're migrating from a monolithic design to a chiplet design, as I said, ideally nothing changes that will make your design much simpler. But the worst case is it's kind of like designing two interfaces to a separate chip that someone else has designed, and you're building a system. You're not able to model the behavior of the other chip necessarily, so that really complicates things. That's why we promote or are currently promoting for a single vendor, Multi-Die Designs, this knock-tunneling approach where you can actually start, if you like, designing your entire multi-die system as a single monolithic design. And then perhaps sometime later, before you start to build RTL. But once you've got the architecture designed, only then do you need to think about how you will physically partition your product into multi-dye system. Excellent. Thank you very much. Moving along here again quickly, we're going to get through a couple more here. This one's interesting. Why isn't a fast die-to-die link enough on its own? What does it take to preserve the behavior and design intent of the overall design? Well, obviously, speed is important. You want to make sure you can get all of the traffic that you have to send across the chiplet link over that link. But you've got to remember that certainly in a network on ship, that communication fabric on ship is used for more than just the data transactions, reads and writes to memory, or communication between different IP blocks. That fabric also transmits information about quality of service to ensure the high priority data packets get through and aren't delayed by lower priority. And as I said, you'll also have information about different power and clock domains so that you have the ability to certainly change clock speeds or shut down clocks and even shut down power and adjust the voltages to save power consumption and adjust the performance. All of that information gets lost when you go to a standard protocol like AXI that's just designed for transactions. But using knock tunneling makes it much easier to transport that information without going to the effort to design out-of-band transport for that. All right, looks like I'm gonna have time for just one more question. Why are AI and other complex workloads pushing SOC designers towards multi-die architecture? And what problems do chiplets help solve? Well, it's all about, in AI, it is primarily driven by the need to put more and more compute into your platform. And you get to a point where larger silicon becomes either impossible, if you want to exceed the reticle limit, the maximum size that you can make a chip, that 26 by 33 millimeters, or it just becomes uneconomic, and it becomes uneconomical long before you get to that big die size. So by breaking the problem up into smaller chiplets that are tightly coupled to each other, you can effectively have much greater silicon area than you could with a monolithic die economics the yield will be much better and you get the reuse uh so you could have multiple different configurations of the chiplets a multitude of reasons excellent well i think that's a great place to wrap up so again to our audience thank you for attending today's webinar for more information related to today's webinar you can visit any of the resource links in your resource folder within 24 hours you will receive a personalized follow-up email with details and a link to today's presentation on demand so thank you again for attending today's 30-minute bitcast connecting chiplets isn't enough solving the data movement challenge in multi-die systems brought to you by ee times and our terrace this webinar is copyright 2026 by aspen core the presentation materials are owned by or copyright by our terrace and the individual speaker is solely responsible for his content and opinion So on behalf of our guests, our terrorists, I'm Colleen Heckman. Thank you for your time, and we hope you have a great day.