Making my life simpler?

Laurie and I started building and hosting stuff on the internet back in the ancient days (before DSL) — plaidworks.com has existed since 1995, and until a couple of years ago, always lived on a server in the house, partly because in the early days, there weren’t that many hosting environments (especially priced for consumer/SOHO) and because like most geeks, we were most comfortable building, customizing and maintaining packages that we used to suit our purposes.

How times change. We’ve not only moved beyond the “host your box here” model, we’re now running head first into the “we have the services you want right here” model.

There are challenges and tradeoffs to all of the models. If you run your own server, you’re responsible for upgrades and updates and security issues and uptime and reliability and…

And the list goes on. I found, after a while, that running your own meant spending most of your time working ON the server, not on the things the server was supposed to be serving. You effectively turn into your own little IT department instead of building content and code — so I happily “outsourced” my services to other places.

But in the back of my head, I thought that might simplify things. In some ways, it does — but in other ways, it simply changes the problems. I’ve been figuring out how I wanted to host things and what the optimum setup was once I roll out Dare2Thrive.

Here’s a diagram of my service network:

that’s not — simple — in any way, shape, or form. Eight vendors (well, technically, a couple of those are yahoo and a couple are google, but still, they’re managed separately). Lots of interconnects and places where you have to worry about compatibility and integration.

In retrospect, I realize I shouldn’t have been surprised. What we’ve done with the maturing of the web infrastructure is move from the model of the build-and-manage geek to a more traditional IT infrastructure. You are really managing vendors instead of code, but the need to manage doesn’t really change. What it changes is the pain points: you have to make sure you have a quality product, but also a quality vendor and one you’re comfortable committing part of your infrastructure to. In many ways, this is the kind of decision that companies make all of the time: the classic being Salesforce vs. SAP vs. Peoplesoft. And once you make it, it’s damn hard to change it, so guess right… (Don MacAskill of Smugmug has one of the best views of this model and how it can benefit you; but he also has some interesting thoughts about what life was like when Amazon S3 burped. It’s no panacea, that’s for sure)

On the plus side, this new model gives you better capabilities at a reasonable price — at a simplistic level, compare the difference between hosting photos with Gallery on a home server and using Flickr or SmugMug or Zoomr. Even at a first approximation, there’s absolutely no way you can afford the network pipes flickr has for an individual installation. Back in 1995, when most content was small and text-oriented, that wasn’t such a big deal. Today? big deal.

Also on the plus side: you get to use the services, not install them, or maintain them, or fix them, or operate them, or upgrade them. It allows you to focus more on content and less on infrastructure.

the negatives? The big one: more things are out of your control. It’s no fun if a service you depend on has a problem — all you can do is file a ticket or make a phone call and wait for someone to fix it. That actually opens up doing this to more people: not everyone are (or want to be) geeks with their arms elbow-deep in code. On the other hand, choose the wrong service, or have a service’s support and quality fall apart over time, and your life can be hell, and there’s nothing you can do.

Other negatives: things can break without warning, especially where you’re using an API or creating an interconnect between two services. Imagine the fun if Typepad did something that broke Feedburner (or worse, created their own version and locked out Feedburner in a turfing fight, like MySpace has done a couple of times). Or perhaps it’s something “simple” like a network problem — your environment is no longer on a single box, or in a single data center, or even a single time zone.

Ultimately, what this has done is turn building online environments from a coding problem (download the software, install it, configure it, manage patches, etc) to an IT problem (define services, discover and evaluate vendors, do the integration, and pray they do their job and allow you to do yours). In some ways, it gives you access to resources and functionality you couldn’t do on your own, but the negative is that more things are out of your control and you have to depend on the vendors you choose to get the job done for you.

What it doesn’t do is make it simpler. Building any kind of environment beyond the simplest one is still a fairly complex beast, but I’m surprised at how well some of these groups have created things that make this job easier (Feedburner — well, I can’t say how nice it is that something like this is so damn SIMPLE, and it just seems to work; I’ve just started experimenting with Yahoo Pipes, too, and all I can say is — good job, folks — it’s a fascinating tool done very well; powerful but not confusing or opaque)

In my case, I was a bit surprised at how complex the environment I decided I needed. It’s this complex because I was keeping it as simple as I could while doing what I wanted done.

But in looking at this, I see some possibilities where organizations could add functionality to their offerings that would allow me to simplify things significantly. I’m going to be moving my email off of hosting matters to Google Apps over time — but if Google Apps supported scripting languages (perl, php, python, ruby, etc) and mySQL as well as static pages, then I wouldn’t need to keep a hosted environment for custom scripting for my sites. If they supported scripting languages, you could potentially run Movable Type or WordPress there, and basically become self-contained using G-A as a hosting environment. Yes, I’ll bet they want us to use Blogger, but that’s not the direction many of us want to go.

Additionally, Google Apps allows me to register a new domain, but doesn’t seem to support taking over existing domains as a registry, so I can’t retire register.com. That seems like a fairly easy one, relatively speaking, for Google to do (and if I missed it, someone tell me!). Google also seems a natural to build a system like Amazon did in S3.

On the Typepad side — again, DNS registry is a missing piece, as is email. Six Apart really has an opportunity here if they choose to take it to add hosting capabilities and take on more of the functionality (hint: I’d do DNS registry first)

Or imagine if Yahoo Pipes added feedburner functionality — or Feedburner adds the pipe munging that Pipes is giving me (what I’m going to use Pipes for: for distinct content pieces, I’m going to create independent blogs to house them, like I did Two for Elbowing and Imaging Reality. Pipes allows me to merge all of those feeds together for someone who just wants a single subscription feed for everything again, simplifying a person’s subscription so they don’t have to think to add or delete a new blog in my universe unless they choose to subscribe to a subset. Very nice hack).

A long time ago in a galaxy far away, Laurie and I actually hosted web sites on our home server; we finally shut it down and told the sites to move to the hosting services, because they could get access to resources (especially the big network pipes) we simply couldn’t support — and then we moved onto the hosting services, too (and I’m actually quite happy with Hosting Matters).

But now, having gone through this analysis and building the architecture, it looks to me like these services are now being challenged for their future. Typepad and Google Apps both seem to be building out environments that are going to act like really large, centralized hosting services, and where they meet is more or less where an organization like Hosting Matters or Dreamhost exists. If you’re a small shop that’s not interested in hosting a box or three in a colo (the Rackspace model), environments like Google Apps will be increasingly attractive.

In fact, the only reason I plan on hanging onto my Hosting Matters environment is for hosting a couple of MySQL databases and a few perl/php scripts (for instance, my email-hiding email archive cloaking script; the raw data will live on S3 behind a password). That, and it’s a nice testing/development platform, although honestly, so’s my Mac Laptop; we’ll see which one I use more over the next year on all of this.

I love TypePad — it’s a great, low-maintenance environment that’s well worth the fees to live there. At the same time, if I could run Perl or PHP on a Google Apps environment and turn it into a hosting environment, it would be tempting to move to Movable Type or WordPress there; it’s a code base to maintain, but it’s fewer systems and interconnects to worry about. A classic IT integration tradeoff to analyze…

I’ll be fascinated to look back at this in a year and see how this stuff matures. I’ll bet I’m down from eight vendors to no more than five — and probably have new functionality, too.

Of course, that won’t make it simpler. Just more powerful (and probably cheaper).

You might also want to read:

1. Is your web site making assumptions about your users? I’m doing a little research for a project I’m working on for someone. Part of that project involves getting users up and running with Eclipse,...
2. Fraser Speirs – Making a living with the camera Fraser Speirs – Making a living with the camera: After hearing various people talk today and various discussions, I made a decision: I shoot photographs...
3. Rethinking how I do things — making decisions (Part 2 of …..) Chuqui 3.0: Web/Tech: So that’s really the basis for what I’m doing now — looking to see what else is part of my “electronic life”...
4. Making blogs more — something… Now that I have a couple of weeks away from the office for the holidays (yippee! yahoo! Yowza! yay!) one of my hopes for the...