Many of the recent great tales of peering, depeering and and repeering involve Cogent (AS174) eventually. This one starts there. Cogent and Sprint (AS1239) established a direct adjacency this week. You may be able to see the little “1239” next to directly connected to the “174” in the picture off to the side. This is big news for a number of reasons, among them:
- Both are big networks
- Sprint is tremendously exclusive in its peering and would almost certainly not offer Settlement Free Interconnection to a network of Cogent’s ilk.
- Cogent is a tremendously cheap (let’s say “cost conscious”) network that would never pay a cent more than they had to for anything
What remains are a whole lot of questions, some of which we have ready answers for and others of which require moles and informants inside the relevant networks. Among the questions are: What kind of peering is it? For what prefixes? In what geographies? Is anyone paying anyone else? Who lost the traffic? Why did Sprint and Cogent do this?
Sprint is, according to our calculations, the largest network in the world. They have the largest customer base with the most reach of any network in the world. Cogent is, unquestionably, a very large network, ranking 11th overall in the world (8th in North America and 6th on the Wholesale ranking worldwide) But Cogent is still only about 1/5th the size of Sprint, according to our calculations. (Renesys calculates customer base by using routing data and using it to determine the size of on-net routed prefixes. The prefixes are then scaled by size. There are fancy algorithms involved but suffice it to say that the data are not traffic data but usually approximate traffic for large networks with a healthy blend of users. More detail on this in the future if there is demand).
So Sprint and Cogent are both big, but Sprint is a lot bigger. Moreover, Cogent has been in several fairly serious peering spats in the past year or two with networks far less picky about their peers than Sprint. Among them Teleglobe, France Telecom and, somewhat famously, Level (3). Most of these disputes appear to have centered on the subject of ratios (Cogent typically sends much more traffic to its peers than it receives). Most large networks enforce peering ratios as a way of ensuring that the costs and benefits of peering are equally shared and, even more subtly but importantly, that the networks that they peer with are similar in structure and business model to their own. Given Cogent’s likely ratios with Sprint, it’s pretty unlikely that Cogent meets Sprint’s settlement-free peering requirements. Sprint doesn’t publish a peering policy to verify that, but I suspect I’m right nonetheless.
Let’s assume it’s paid peering. This really doesn’t matter, other than a point of curiosity about Internet economics as they evolve. It’s still a routable adjacency that changes the way traffic moves between two big networks, and therefore is interesting. Traffic used to move between Sprint and Cogent via NTT America (AS2914, formerly Verio), and some still does. Let’s look quickly at what is being routed where in which directions in order to characterize the new adjacency between Sprint and Cogent.
As of 30 October at 00:00 UTC (picked as a useful boundary several hours after the peering came up on 29 October at about 17:30 UTC), here’s what things look like. (Remember, BGP paths should be read right-to-left when thinking about routing originations and transitings, and left-to-right when thinking about traffic flows).
- 1239_174 (originated by Cogent, carried by Sprint): 3478 prefixes
- 1239_2914_174 (originated by Cogent, transited by NTT America and carried by Sprint): 231 prefixes
- 174_1239 (originated by Sprint, carried by Cogent): 4631 prefixes
- 174_2914_1239 (originated by Sprint, transited by NTT America and carried by Cogent): 127 prefixes
As an interesting note, there were a lot more prefixes announced by Sprint and carried by Cogent (174_1239) briefly (about 2300 of them) but most of those went away within a few seconds.
So Cogent is selecting more routes directly on Sprint’s network than Sprint is selecting on Cogent’s network, but they are roughly of the same order. And there is no simple way to predict traffic levels directly from this information. The next interesting question is: what prefixes are these. A quick analysis of a basic registry geolocation of the prefixes makes it fairly obvious: between 70 and 85% of the prefixes (in both directions) on the new edge between Cogent and Sprint are in the United states. And virtually all of the prefixes routed on the edge including NTT America are outside of the United States.
What conclusions can we draw from this? What is the “cash value” as Thomas Pogge used to always say? Cogent has paid, North American peering from Sprint and is routing their European traffic to Sprint over their existing Transit connections with NTT America. It’s possible that things are messier than this (given that less than 100% of the prefixes carried can be verified to be North American, and given that there may be differences in the way traffic is routed in each direction), but this is almost certainly the basic story.
We return to the question of why. If I’m right and this is a new, paid peering between Cogent and Sprint, why could Cogent pay Sprint for this interconnection? The answer is fairly obvious: because Cogent got a better deal from Sprint for the traffic they need to send to Sprint’s network than they were previously getting from NTT America. And furthermore, they realized that Sprint was never going to give them a Settlement Free Interconnection, so there was no harm in paying for it, if the price was right. See my point above about Cogent being an extremely “cost conscious” network. No other explanation is necessary. Note that “better deal” does not necessarily mean “cheaper price”. There are a variety of factors that go into calculating the cost/benefit equation of an interconnection and price per megabit per second is only one of them.
As a final note: there was a funny conspiracy theory spun on the NANOG mailing list about this issue. The general theory put forth was that the entire story is about Deutsche Telecom (DTAG, AS3320). Unlikely, you say? This is the very nature (and appeal) of conspiracy theories. The story goes something like this: Cogent traffic to DTAG is congested. DTAG refuses to upgrade the peering because they think it hurts Cogent more than it hurts them. DTAG buys from Sprint and Cogent knows this. Cogent gets paid peering from Sprint so that they can force DTAG traffic down that link and make DTAG pay for it. It’s a nice theory but it’s not realistic.
While it is true that DTAG buys from Sprint, it is not true that a network of Cogent’s size would make a paid peering decision purely on the basis of one congested peering. Moreover, the rumors are flying fast and furious that lots of DTAG peering is congested right now, not just to Cogent. I have no personal knowledge about whether that’s true, but it’s becoming a persistent story. Although I don’t doubt for a second that many European networks would like to reduce the quality of their peering with Cogent out of anger over the way Cogent has dropped prices in the European market, there’s no evidence that DTAG’s peering congestion with Cogent (if it exists at all) is caused by that anger. And there’s even less evidence that a network like Cogent would pay Sprint for peering just to spite DTAG. It is a fun theory, though.
Full disclosure: Lots of the networks discussed here are Renesys customers. When I write about them, I do so without using any information they may have disclosed to me as customers, but only what my coworkers and I can gather from the routing tables that they publish to the world. And I try to give them the same scrutiny and critique that I give any other network.