I don't know if anybody really gets any value out of the stuff I write here, but I think it's important, and interesting, and good to know if you're interested in the nature of the Internet.

In recent years, a service called Twitter introduced a message distribution system that lets people send short messages over the internet to people who have subscribed to those messages. By short, I mean 160 or so characters. The overall quality of information one can transmit in 160 characters has been a criticism of the service, but people continue to use it in the interest of having access to short thoughts from their friends, politicians, celebrities, and a number of businesses who find interesting ways to use the service.

Out of Twitter came the necessity for URL shortening services. URLs (e.g., http://idkfa.com) (idkfa.com)) are human readable "resource location" addresses, or at least, were initially designed to be human readable. Their initial intention was to define a) the protocol over which you were accessing a file, b) the host which was supposed to serve you the file, and c) the resource location on that host of where you expect to find that file. Most web sites only make you memorize the host name, providing default pages at the "/" resource location, and most web browsers don't require you to enter in the http://. Even if efforts are made to make it easier for humans to enter URLs (bookmarks, URL completion), there's still the problem that most URLs (after they've been "helped" by the computer) are much longer than 160 characters (even much, much longer for sophisticated and intricate web applications).

Thus, Internet companies invented URL shortening services. These services would take a long url (say, http://idkfa.com/v3...id=1&msg_id=1) and turn it into something like http://bit.ly/g3WVDN (bit.ly) (from 54 characters down to 21).
If a person clicks on the bit.ly link, they are taken to the bit.ly web site, which then redirects them to the longer URL you intended. This is ideal when you are trying to provide both a comment on something, and then a link to what you're commenting on in your limited 160 character Twitter message.

Now, in almost all Twitter clients, *all* of your URLs are shortened automatically, even when it's not necessary in order to fit in the 160 character mark.

However, the concept of URL shortening is not altogether altruistic. Every time you click on a "shortened" link, you access an unrelated site first before you're redirected to the content you're actually interested in. Why is this bad, Josh? Do the URL shortening services try to send you to the wrong place? Do they try to forward you to viruses, or other malicious content?

Not in the least. Their entire livelihood in running a free service is being reliable, accurate, and trustworthy. And because the system is so simple, it takes almost nothing to switch from one URL shortening service to another. The URL shortening business is a business in hopes that you maintain a habit.

So how, then, does a URL shortening business make money? Particularly, the large services, for which huge amounts of bandwidth are passed through for absolutely no cost to the users? How do they keep the lights on?

They do so by identifying trends. That is, on their side of things, they have a massive database of URLs that people are clicking on. And based on those URLS, they have identify patterns in usage. Things like:

• How many people are visiting a particular site?
• What types of content are they viewing?
• Is this more popular today than it was yesterday?
• What are people interested in? And what alternatives are they not interested in?

It is essentially free, accurate, and real-time market research for anyone who wants to buy it. The information tends to be anonymous, and thus, generally has no personally identifiable information associated with it to get into sticky privacy messes. It's a data mining treasure trove.

Businesses have of course caught on. The original URL shorteners are now competing with companies like Facebook (their fb.me service). It's a subtle, but pretty lucrative market. And it's all for the ability to identify trends before people realize they're trends in the first place.

What's more fun is that this is completely old-hat, in terms of companies using "clicks" or recording usage to get data. Search companies like Google and Yahoo have been identifying trends since they realized how accurately people's search terms correlated to various social, economic, and political trends.

The Internet is a weird and scary place. And the big players on the Internet know exactly what you're thinking.