The difference between overlay networks and underlay networks still causes issues with engineers everywhere. I keep trying to find a visualization that boils everything down to the basics that everyone can understand. Thanks to the magic of online ordering, I think I’ve finally found it.
Candygram for Mongo
Everyone on the planet has ordered something from Amazon (I hope). It’s a very easy experience. You click a few buttons, type in a credit card number, and a few days later a box of awesome shows up on your doorstep. No fuss, no muss. Things you want show up with no effort on your part.
Amazon is the world’s most well-known overlay network. When you place an order, a point-to-point connection is created between you and Amazon. Your item is tagged for delivery to your location. It’s addressed properly and finds its way to you almost by magic. You don’t have to worry about your location. You can have things shipped to a home, a business, or a hotel lobby halfway across the country. The magic of an overlay is that the packets are going to get delivered to the right spot no matter what. You don’t need to worry about the addressing.
That’s not to say there isn’t some issue with the delivery. With Amazon, you can pay for expedited delivery. Amazon Prime members can get two-day shipping for a flat fee. In overlays, your packets can take random paths depending on how the point-to-point connection is built. You can pay to have a direct path provided the underlay cooperates with your wishes. But unless a full mesh exists, your packet delivery is going to be at the mercy of the most optimized path.
Mongo Only Pawn In Game Of Life
Amazon only works because of the network of transports that live below it. When you place an order, your package could be delivered any number of ways. UPS, FedEx, DHL, and even the US Postal Service can be the final carrier for your package. It’s all a matter of who can get your package there the fastest and the cheapest. In many ways, the transport network is the underlay of physical shipping.
Routes are optimized for best forwarding. So are UPS trucks. Network conditions matter a lot to both packets and packages. FedEx trucks stuck in traffic jams at rush hour don’t do much good. Packets that traverse slow data center interconnects during heavy traffic volumes risk slow packet delivery. And if the road conditions or cables are substandard? The whole thing can fall apart in an instant.
Underlays are the foundation that higher order services are built on. Amazon doesn’t care about roads. But if their shipping times get drastically increased due to deteriorating roadways you can bet their going to get to the bottom of it. Likewise, overlay networks don’t directly interact with the underlay but if packet delivery is impacted people are going to take a long hard look at what’s going on down below.
Tom’s Take
I love Amazon. It beats shopping in big box stores and overpaying for things I use frequently. But I realize that the infrastructure in place to support the behemoth that is Amazon is impressive. Amazon only works because the transport system in place is optimized to the fullest. UPS has a computer system that eliminates left turns from driver routes. This saves fuel even if it means the routes are a bit longer.
Network overlays work the same way. They have to rely on an optimized underlay or the whole system crashes in on itself. Instead of worrying about the complexity of introducing an overlay on top of things, we need to work on optimizing the underlay to perform as quickly as possible. When the underlay is optimized, the whole thing works better.
