Regarding PRISM, and the "whistleblowing" of Edward Snowden, exposing the NSA in their ability for warrantless surveillance:

This sort of thing is, of course, fascinating to me. It at once affirms my paranoia, my distaste for "the cloud," and my overall fear of government-sponsored computer espionage.

This item is not new, however. This type of technology has existed and been in place since 2007 (www.pbs.org) (and likely long before). While acknowledged in court, it had failed to find any significant public attention (discarded at one point in the interest of security, I think).

The conversation about the legality, ethics, etc., is a lengthy one, and probably one I'm not really equipped to argue. However, I can comment on some of the interesting technical pieces. For instance, this diagram:

As the horrid PowerPoint rounded box says, traffic on the Internet follows a least cost path. The cost of a path from one point on the Internet to another is a metric measured in time, error rates, and manual overrides to determine which paths will be taken given a source and target.

For example: you're an Internet router, and you have a choice among routes A, B, or C. A is fastest, so it usually wins out over B or C. However, if A goes down (fails), it chooses among B or C until the cost metric for A can be established to be cheaper than B or C. This makes the network failure resistant, meaning that it can route around failures without human intervention.

The above example works when you replace A, B, and C with names of countries (or continents). A small African country wants to route a packet to Latin America. To route this packet, it looks at the metrics for the routes it has available. For the diagram above, it has the choice among a 343 Gbps link to Europe, 11 Mbps link to North America, and 40 Gbps to Asia. However, the metric for cost is for the full path, not just the "ways out." It will take into account the massive capability of the North America/Europe link, as well as the North America/Latin America link. Rather than the Africa ->North America -> Latin America route, it will take the Africa -> Europe -> North America -> Latin America route because it is determined to have the lowest total cost.

The PRISM application (and those like it) exploit the fact that for most inter-continental traffic, the US and/or Europe ending up being the lowest cost route, and thus, pass traffic along. Being able to collect, aggregate, and analyze this data is made possible purely by the fact that our external Internet infrastructure (note: not necessarily our internal, Internet-to-your-home infrastructure) is considerable. Other continents route through us because we are cheaper.

A fun part of this is that the route metrics I describe above can be manipulated. The routing devices I describe are all configurable devices, and the speeds/metrics they advertise to their neighbors is purely configuration not determined by any physical or hardware limitation. When they set up the devices, they input the values that say "I can handle this speed." Which means that it can be changed, and other routers have no choice but to route their data accordingly.

China has done this one multiple occasions, during which they will advertise things like, "We have better speeds, so your routes should use us instead," during which if we do not have the routes statically configured on our end, we will happily route all of our data through China.

This exploit is purely based on how the Internet was designed, and isn't a flaw (unless routing metrics are lied about). Aside from being installed at the right "observation" points, the US has always had this ability, as we are the "default" route for much of the Internet. When the Internet is passing through you, there's value in seeing from where and to where things are going, and depending on the case, may have data mining or intelligence significance.

Weirdly, thinking on this a bit more, I've actually built systems like this before. At one point, our customers asked if instead of having to call into us, we could provide them with a "top talkers" report (a listing of "Who is pumping the most traffic over the last N minutes?") This information was only available by way of the programming interfaces on our devices, which we would never give up to our customers, but we still needed to find a way to provide this information without wasting time on phone calls.

And so, we had our routers report a stream of data back to a centralized server, where it could then be received, analyzed, and made available to our customers via a secure portal. Our coding ensured that all customer data was strictly separated, and only a certain number of customers were even provisioned to have this feature.

The data itself wasn't particularly interesting. At least, not in the details. However, when combined over time, we could see things like malicious botnets, DDoS attacks, iTunes, Youtube, etc. (whatever was using the most traffic at the time). Being that we were only telling the customers what was going on on their own network, there were no ethical questions raised (aside from those like me being able to see the data).

The mechanism here would have been exactly the same. They may not be able to see content, but they can see where you are coming from, to where you are going, and however much. It would not seem important at small details, but given the granularity of searches or the intelligence of the pattern recognition, you could very well catch things like news events, uprisings, etc., as they are happening.

Also, fun side note: the Read/Unread mechanism on idkfa was used on the "Top Talkers" report to be able to track/discard/identify new messages versus repeat messages.