As I said in my INFRASTRUCTURE! post, I’m fascinated by the way things work.  This continues on to my fascination with email;  such a simple protocol, and so important to our lifestyle!

This post on the InnerLayer was my longest ever (including my short story), where I discuss email and how awesome it is…

“You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles. Do you understand this? And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.”

— Albert Einstein, explaining radio.

Email, like Telephones, the Internet, Power Lines, Credit Card Terminals, ATMs, etc. have become so much part of our daily lives that we just accept it without really thinking about it. You click “New,” put some data into a form, then click “Send.” In a couple of seconds (minutes if your mail servers are suffering a spam blizzard), your LOLCAT Email has zipped from one end of the Internet to the other (This proves that, unlike radio, Email does have a cat). Do we marvel at this paragon of technology? Not really. I generally grumble at the mixed blessing that has brought me spam.

A while back, I made the mistake of reading the Wikipedia article on the Power Grid. I was amazed at the mechanics of the system that allows a handful of uranium melting itself into slag in Glen Rose, Texas to shove electrons across a couple hundred miles of copper, through a handful of coils… all to be stopped by a little switch in my wall before they could excite the gasses in my Compact Fluorescent light bulbs. For two weeks after that, I was a road hazard. In the middle of my commute, I’d glance at my right hand mirror, gaze out my side window, and think “Hey! I didn’t know they were running two phase power out here!” I must say that I couldn’t have been a more dangerous driver during those two weeks if I was dryping.

My latest project at SoftLayer (WHOIS management for ARIN… look in the portal under SWIP) had me doing research on Email (as all transactions are conducted through Email).

Like David Bowman staring into the black monolith at the end of 2001, I was struck by the simplicity and beauty of the Email system!

Let’s tag along with an Email message, so you can see just how cool this is.

Bob is currently taking a vacation in Glen Rose, Texas (taking the tour of the Nuclear Power Plant). Sitting in the lobby of the Visitor’s Center (thinking about this giant reactor being at the beck and call of his little switch), he has a firm wish that Alice could come see this. He’s a cheapskate, though, (that’s why he’s at a local power plant instead of, say, Disneyworld) so he’s not going to waste money on a Power Plant Postcard and stamp at the gift shop. No sir, he opens up his blackberry and sends a short Email to Alice (”Wish you were here!”). A few seconds later, Alice’s Email client pops up an alert that she’s received an Email from Bob. The Email itself is simply a block of text… really! No magic going on here! Here’s what the Email looks like:

From: bob@example.net
To: alice@example.com
Date: Wed, 10 Oct 2007 12:54:08 -0400 (EDT)
Received: From EXAMPLE.NET
 by EXAMPLE.COM with SMTP
 Wed 10 Oct 2007,
 12:55:00 -0400 (EDT)

Wish you were here!

---
Bob
Sending to Alice for over 30 years.

Simple, huh? To help visualize this transaction, Figure 1 shows a “RFC 2822 Compliant Post Card:”

Bob opens up his Email client, and writes “Wish you were here!” in the message area. He then adds his signature, and writes in the subject. The Email client adds headers to the address area of the Email (From, To, and Date), then drops the Email into Bob’s local Email server (EXAMPLE.NET).

The Email server at EXAMPLE.NET looks at the address section of the Email and notices where it is to be sent to Alice (who has an Email box at EXAMPLE.COM), and opens up a SMTP (Simple Mail Transfer Protocol) connection to Alice’s mail server. It then sends the message and moves on.

Alice’s Email server puts a “postmark” header on the Email (the Received header) and drops a copy of the entire text block into Alice’s Email box.

Alice opens up her Email client, which then downloads all the text files from her Email box. It reads the From header and tells Alice that she has an Email from bob@example.com

That’s it! Simple, huh?!?

If Bob’s Email server can’t send the mail directly to Alice Email server, it can send the Email through a relay server. This server adds its own “Received header” to the Email. If you look at the headers of any Email you’ve received (spam is not only a good source of vitamin Sodium Nitrate, but also an excellent resource for Email headers), you can see every single server your Email passed through. It’s like those neat stickers customs officials stick to your luggage as an apology for cracking your locks when you fly internationally.

That’s all there is to it! A simple block of text, passed off to an Email server. And the actual protocol is just as simple. Here’s what the communication between Bob’s Email server and Alice’s Email server looks like (modified from the example on Wikipedia’s SMTP article):

Bob’s server connects to Alice’s and identifies itself:

ALICE: 220 smtp.example.com ESMTP Postfix
BOB:   HELO example.net
ALICE: 250 Hello example.net

Bob’s server then tells the receiving server about the Email:

BOB:   MAIL FROM:<bob@example.net>
ALICE: 250 Ok
BOB:   RCPT TO:<alice@example.com>
ALICE: 250 Ok

Bob’s server then tells Alice’s that it’s ready to send the real message:

BOB:   DATA
ALICE: 354 End data with <CR><LF>.<CR><LF>

Next follows the RFC Compliant Email message from above, ending the data with a “.”, which tells Alice’s server that Bob’s message is complete:

BOB:   From: bob@example.net
BOB:   To: alice@example.com
BOB:   Date: Wed, 10 Oct 2007 12:54:08 -0400 (EDT)
BOB:   Received: From EXAMPLE.NET
BOB:    by EXAMPLE.COM with SMTP
BOB:    Wed 10 Oct 2007,
BOB:    12:55:00 -0400 (EDT)
BOB:
BOB:   Wish you were here!
BOB:
BOB:   ---
BOB:   Bob
BOB:   Sending to Alice for over
BOB:   30 years.
BOB:   .

Alice’s server lets Bob’s know that the message is queued to go, and Bob’s server signs off:

ALICE: 250 Ok: queued as 12345
BOB:   QUIT
ALICE: 221 Bye

I marvel at this technology. Every Email in the world is transmitted by this simple protocol. The whole of electronic communication takes place by handing small blocks of text from one Email server to another, until it finally makes its way to the recipient’s inbox. That’s all! No magic potions, no hocus pocus, no tying messages to carrier pigeons or pulling cat tails.

Not only this, but your message is flying through a blizzard of spam. Because the protocol is so simple, people build simple tools that blast out millions of messages at a time, flying all over the Internet. But the awesomeness of this just makes Email that much more awe inspiring. Email has been running, nearly uninterrupted (as a whole) for DECADES under the most concerted distributed denial of service attack of all time.

And STILL your Email gets to its destination. Benjamin Franklin would be proud.

Think of this the next time you forward the latest list of funny jokes to everyone on your Email list. This incredibly simple protocol will make sure that your vital Email gets to every recipient listed! “Neither snow, nor rain, nor heat, nor gloom of night stays these couriers from the swift completion of their appointed rounds…”

Originally posted on Tuesday, October 16th, 2007 at http://theinnerlayer.softlayer.com/2007/the-miracle-of-email/.

I’m one of those nerds who get excited about INFRASTRUCTURE!  Web applications are exciting and fun… but I like looking in the blogs to see how they wired up the servers and wrote the programs that DO what the application does!

I assisted with SoftLayer’s “Virtual Private Racks” (an infrastructure upgrade that lets you pool server resources as if they were wired into one rack and one router, no matter where the servers were physically located) and the SoftLayer API (a SOAP api that lets you do EVERYTHING on the SoftLayer network)… and both projects are all about INFRASTRUCTRE!

So… my InnerLayer blog entry about INFRASTRUCTURE:

Wal-Mart! Champion of Retail! Who else can build a large brick box, paint it blue, stuff it with stuff, and make money hand over fist? What is the source of this power? Many will say it’s their sheer size. However, this isn’t true! Because what many people forget is that Wal-Mart had to start with one single store, just like every other retailer in America. So what is their secret?

INFRASTRUCTURE!

It’s been said that Wal-Mart can track a single apple from the tree to the front of the store. Every piece of inventory is logged and tracked from pickup to delivery. Every single bottle of aspirin, every sock, every donut is duly logged and mashed up in massive data warehouses where giant computers munch the data and produce useful reports. You know what the most popular item is at Wal-Mart? According to an employee friend of mine, in the Cedar Creek Lake Area of North Texas, it’s Bananas. They know how many bananas are sold, when they were sold, what the best day of the week is for banana sales, and which cashier is responsible for the most banana sales during a month. They can track banana sales over time, by store, region, trucking company, banana producer, you name it. They know which employee was on duty in the fresh fruit aisle when banana sales were high, and which employee used to be on duty in the fresh fruit aisle when banana sales were low. It’s all in there, if you want it.

However, Wal-Mart had to build this technology from scratch. They had to install special data systems in their distribution centers. They had to build their own server farms, lease their own data lines… did you know Wal-Mart has it’s own SATELLITE NETWORK?!? The Wal-Mart Satellite Network is one of the largest private satellite systems in the world, carrying real time data from every single Wal-Mart store and distribution center to Wal-Mart’s headquarters in Bentonville, Arkansas, where it is poured into their massive data warehouse. They can plan, instantly, to take care of overstocks and shortfalls at every store, as soon as it happens.

You don’t need to build your own satellite network to get competition crushing infrastructure today. Using the technology solutions provided by SoftLayer, and simple connections to the Internet, you too can have the type of infrastructure necessary to succeed in today’s business world. We provide world class servers for your number crunching, huge amounts of networked storage for your data warehouse, geographically diverse datacenters for disaster security, and a private network that allows you to tie it all together as blazing high speeds. Using our awesome API 3.0, you can automate just about every part of maintaining your infrastructure. Leveraging the Internet, you can build data portals that allow your partners to keep you up to date on production, to plan finances, track bananas, whatever you want to do!

We’ve already taken care of the hard work required to build the infrastructure. Now all you have to do is leverage it.

 

Originally posted on Thursday, April 17th, 2008 at http://theinnerlayer.softlayer.com/2008/infrastructure/.