Rich Whiffen

There was a time when Thursday nights were ‘must see TV’ nights. Friends, Seinfeld and ER made a pretty compelling night of ‘must see TV’ as the slogan went. A significant portion of the nation would be sharing the same experience Thursday nights. I remember getting up early on a Saturday, even though it wasn’t a school day because that’s when the good cartoons where on. I doubt my kids will ever have that notion or experience. Between DVR/TiVo, On Demand broadcasts, Web delivery and AppleTV there’s not as big a driver to sit down at a scheduled day and time and watch.

For the past few years we only had terrestrial broadcast TV and an AppleTV in the house. One time, while watching Arthur on PBS, Renee had to go to the bathroom and was jumping up and down demanding we ‘pause it!’, not understanding that not all TV shows were like the AppleTV version. A few months back we added Cable to the house and a DVR unit, so now we can pause live TV, further blurring the distinction between the on demand vs on schedule showings. She doesn’t understand why she can’t watch Zula Patrol any time she wants and has to wait until 7:30 to see it. She’s convinced its something that I’m not doing for her, and not a case of it not being available On Demand or via AppleTV. Further there’s only one episode to watch and when it’s done it’s done. With all her other shows there’s always another episode, so she asks to watch another Zula Patrol, and i have to tell her no. From her perspective it’s no different then saying she can only watch one Super Why. We have more, I’m just not allowing her to watch them. Except in this case, we legitimately don’t have more to watch. Kind of works in my favor I guess, there’s no chance I’ll cave in and let her watch a 2nd one.

As more and more entertainment options become less tied to the providers schedule and less tied to the TV as the only way to watch it, the notion of a good night for TV will wither away. There will still be some notion of scheduling, but it’ll be the date and time it’s put on the distribution network. It probably won’t be the same though. Even for me it’s not quite the same. I love CBS’s monday line-up, but I don’t think I’ve watched any of them at there actual broadcast time in 2 or 3 years. I don’t think TV’s dead or going away. As Randall Hounsell put it, TV is still “a lean back experience.” People will still want to get someplace comfy and be immersed in a world that isn’t there own.

Back when CD’s were the norm, but vinyl records were still around, my over-used joke used to be that my kids were going to be asking me “Dad, how do we get this big black disc into the CD player?” Now I’m not so sure they’ll even remember what a CD is. Never thought the same would happen for TV. The Qwest commercial from the late-90’s is finally coming true.

A tired man goes into a cheap motel in the middle of nowhere and asks about amenities. When he asks about entertainment, the girl responds “all rooms have every movie ever made in any language anytime day or night.” It’ll probably be 20 years after the fact, but it’s coming.

Yeah, yeah, I know… nobody but me seems to care. Anyway, it seems google has decided to update ‘Google Voice and Video” to version 1.0.19.1554 and “Google Software Update: to version 1.0.7.1306. What’s interesting about the 2nd one is what happens now when you try to launch the old “Google Update” application:

You’re greeted with this:

Only one option “Get More Google Software” which isn’t what I asked for, I asked for Google updates. Ok, no problem, I’ll just quit the app:

Or maybe I won’t since they’ve taken that choice (and the ‘about’ option so I can’t tell they’ve updated it with out my knowledge perhaps). Oh well, lets take the only path given to me and lets “Get More Google Software”

Oh fun! Three new windows. One expected, two not. The google web page with mac software, a dialog box because finder needs permissions, and an disconcerting ‘preparing to move to trash’ dialog box. I just wanted to see if I was running the latest version of google earth and now I’m given this? Worse yet, is there’s not much context for the permissions request:

As before, it’s not so much about the actual changes google has made to my machines, but the lack of permission they sought from me to do it. This one is particularly troublesome for me because they don’t really give me any choice but to take their path and that path isn’t clearly explained in their dialog boxes.

I’ve started this rambling post four or five times. Deleted it every time. So I’ve decided to take a different tact. Rather than writing a long post that says everything, I’m going to write pieces of it. This way I’ll actually get something written. As the Japanese proverb goes “Vision without action is a daydream. Action without vision is a nightmare.” All too often I’ve run into folks who are long on vision but short on action. A few times I’ve been in organizations that are restructuring to better cope with the current environment. The ever present ‘re-org’. One of those was the transformation into a service delivery organization. Which was a good idea and there was good vision behind it. The action is where the idea died and cost people their jobs.

I’ve had this picture hanging around for a few years. I stole it from an issue of eWeek. It was from one of those articles that is really nothing more than a advertisement in essay form.

This picture does a pretty good job at explaining what I mean. Although it’s not the ‘buzz word’ it used to be, being a ’service based organization’ was the goal of a lot of IT organizations. On paper it looks great. It can be an effective way to run an organization. Unfortunately the trick is getting from where you are to where you want to be. It’s been all to common for organizations to ‘green field’ the new way of doing things and make a sweeping change to transform into the desired structure in the shortest possible time frame. And it’s usually a disaster for the first two quarters. There’s a lot of uncertainty on how things get done or who does them. Process bottlenecks creep up everywhere. There’s inconsistency in implementation between teams. And while all of this is going on, real work needs to get done to keep the business going. After a while people start to revert to the old way of doing things or a hodge-podge in between the old and new.

The step that gets missed is the transition and how much transition can be achieved in one fell swoop. If you’re currently a ‘turmoil’ or ‘reactive’ organization and you want to be a service-based organization, it’s unrealistic to jump right to the end state. With out learning the lessons that come with being reactive, it’s difficult to be proactive. If an organization doesn’t have a solid proactive foundation, it can never be service based. Worse yet, there are budgetary considerations that go along with crossing over from one level to another. Software and hardware tools are often needed to achieve the desired state. Although often over looked in the planning stages, it’s possible to make up that budgetary gap. Another gap that’s overlooked is the people side of things. I have never seen an organization budget staff time and overhead to these types of changes. It’s always expected to be done in the margins after a one or two hour ‘training course’ that typically just reads the new process aloud to everyone in attendance. No attention is paid to how to get the staff to the end goal. No real-world examples provided for how things should work. No governing authority to turn to for guidance. No one to find parts of the organization that our floundering in the new process/structure and pitch in and help them through it. Proud in their new organizational structure and plan, leadership pass it down the chain, with implementation left as an exercise to the reader.

So as I try to manage my team, I’ve tried to utilize some of the failure lessons I’ve learned. I don’t make broad sweeping changes if avoidable. There always needs to be a balance of course, you don’t want to make hundreds of small course corrections when a few larger ones will be as effective, but I lean towards the smaller changes. I plan in the overhead. If I’m going to add new processes or procedures to my staff’s duties, I adjust time expectations accordingly. An example would be our post-mortems on outages. I wanted to change how that was done. It should be a 30 minute meeting, but because people were new to it, the first few where schedule for 60 to 90 minutes and we brought in lunch. Walk everyone through the new process a few times. Going back to the post-mortem example, we talked openly about the process and the actual problem in the same context. Giving people a new process with out a concrete example to work with leaves things to interpretation, and you’ll get as many interpretations as you have staff members. By walking through it a few times with everyone they all here the same questions and my answers to those questions. It’s not perfect or without flaws, but it seems to be working.

So I’m having my first run-in with a DotHill Array (rebranded as an HP product). The HP model number is MSA2012fc but the DotHill number would be a 2730. It’s your typical looking 3U 12 disk array. It has two controllers and two 4Gb uplinks per controller. Not too bad. It only does whole-disk raid sets however so it’d be a little silly to plug it into a SAN switch, but it can done. The HP web interface is pretty straight forward and has simple concepts, you create a vdisk, which is a raid set of drives and then you carve off chunks of that vdisk to present as LUNs to the hosts. Similar to the old school Clariion’s, it has the notion of assigning a lun to a particular controller. So you have to manually/mentally balance your workload. It does provide some fairly comprehensive performance stats which can help in that regard and there’s a command line interface w00t!. So far my results with these devices hasn’t been good at all.

First off, I should point of we have two of these. One works, one doesn’t. On the one that works, when I run a simple Bonnie++ test I get results like this:

Not very good at all, especially considering it has 4GB of cache and this is the only test running. I mean 25145ms worth of latency for the block output? Yikes!

On the second one, it seems to work ok. I can do some “dd if=/dev/random of=somefile” of arbitrary size and it happily chugs along. But if you try to run a Bonnie++ run against it you get a lot of errors:

Jul 21 10:09:36 oracle-dev-02 kernel: : exe="?" (sauid=81, hostname=?, addr=?, terminal=?)'
Jul 21 11:00:50 oracle-dev-02 kernel: sd 0:0:0:1: SCSI error: return code = 0x08000002
Jul 21 11:00:50 oracle-dev-02 kernel: sda: Current: sense key: Aborted Command
Jul 21 11:00:50 oracle-dev-02 kernel:     Add. Sense: Scsi parity error
Jul 21 11:00:50 oracle-dev-02 kernel:
Jul 21 11:00:50 oracle-dev-02 kernel: end_request: I/O error, dev sda, sector 934774511
Jul 21 11:00:50 oracle-dev-02 kernel: Buffer I/O error on device sda1, logical block 116846806
Jul 21 11:00:50 oracle-dev-02 kernel: lost page write due to I/O error on sda1
Jul 21 11:00:50 oracle-dev-02 kernel: Buffer I/O error on device sda1, logical block 116846807
Jul 21 11:00:50 oracle-dev-02 kernel: lost page write due to I/O error on sda1
Jul 21 11:00:50 oracle-dev-02 kernel: Buffer I/O error on device sda1, logical block 116846808
Jul 21 11:00:50 oracle-dev-02 kernel: lost page write due to I/O error on sda1
Jul 21 11:00:50 oracle-dev-02 kernel: Buffer I/O error on device sda1, logical block 116846809
Jul 21 11:00:50 oracle-dev-02 kernel: lost page write due to I/O error on sda1
Jul 21 11:00:50 oracle-dev-02 kernel: Buffer I/O error on device sda1, logical block 116846810
Jul 21 11:00:50 oracle-dev-02 kernel: lost page write due to I/O error on sda1
Jul 21 11:00:50 oracle-dev-02 kernel: Buffer I/O error on device sda1, logical block 116846811
Jul 21 11:00:50 oracle-dev-02 kernel: lost page write due to I/O error on sda1
Jul 21 11:00:50 oracle-dev-02 kernel: Buffer I/O error on device sda1, logical block 116846812
Jul 21 11:00:50 oracle-dev-02 kernel: lost page write due to I/O error on sda1
Jul 21 11:00:50 oracle-dev-02 kernel: Buffer I/O error on device sda1, logical block 116846813
Jul 21 11:00:50 oracle-dev-02 kernel: lost page write due to I/O error on sda1
Jul 21 11:00:50 oracle-dev-02 kernel: Buffer I/O error on device sda1, logical block 116846814
Jul 21 11:00:50 oracle-dev-02 kernel: lost page write due to I/O error on sda1
Jul 21 11:00:50 oracle-dev-02 kernel: Buffer I/O error on device sda1, logical block 116846815
Jul 21 11:00:50 oracle-dev-02 kernel: lost page write due to I/O error on sda1
Jul 21 11:00:50 oracle-dev-02 kernel: Aborting journal on device sda1.
Jul 21 11:00:50 oracle-dev-02 kernel: __journal_remove_journal_head: freeing b_committed_data
Jul 21 11:00:50 oracle-dev-02 last message repeated 5 times

Wee fun! Even better is it corrupts the disk beyond recognition. You can’t even fsck it in a reasonable about of time. It’s faster to reformat. We’ve had HP on site and opened a a case or two on this but so far have gotten no where. For me the biggest issue is I don’t have enough time to sit on hold with the call center while they find someone who knows how to use linux with an MSA.

So far I’ve tried swapping HBA’s, cables, ports, fibers. Updating all the firmware and drivers (the MSA, the HBA, OS, etc). I’ve tried different versions of RHEL 5.3 and 5.2. I’ve tried only using the HP supplied drivers, firmware and utils. All with the same results. Later this week, I’m going to give in and install win2k3 or win2k8 and run something like Bst5 or iozone and hope I can reproduce the error. Under low loads it doesn’t error out. It performs poorly but doesn’t error out. I hate these kinds of problems. There’s obviously something wrong with the array, since one works and the other doesn’t, but it passes all the diags. At some point this thing’s going to end up like the printer in Office Space.

Back in my university days I was all about Linux. My first ‘machine’ was a 386sx, probably 16Mhz or so and booted of a 5.25″ floppy. Having to compile the kernel every time you wanted to make any kind of changes and then ‘rawrite’ it out of the floppies. And forget about package management (well until slackware for me…) My first ‘workstation/server’ that I seriously used, scuba.uwsuper.edu, was a 486DX-50 with a Cirix CPU around 1992. I think it might have had 256MB of Ram and a 80gb Segate drive (3.5″ form factor no less!). I think some archive.org listings for the web pages I used to host on it are still around, although from near the end of my use of it. http://web.archive.org/web/*/scuba.uwsuper.edu Good times…

Then I moved to the DC area and started working with Sun and AIX hardware. Linux moved to a novelty/side item for me. RedHat, back when you could run it and not pay for it if you didn’t want to. I’d have a 2nd PC in my office, mostly to act as my X server for working with the Sun boxes more than anything else. At Convergys and Red Cross, we had linux. A fair bit of it too, but in most cases it was never the ‘core’ of the product/platform offerings.

Well my current job with StreamSage is primarily a Linux shop, in particular a RedHat shop (Comcast, the corporate parent is a large RedHat customer). So it’s been an interesting time getting back into the swing of things. On the one hand, I really like getting back into the linux state of mind. On the other hand, I’ve really come to appreciate the work that has been done in AIX and Solaris in terms of hardware management, diagnostics and configuration. I mean there are Linux equivalents in a lot of cases and a lot of it is an artifact of the hardware and software being built by the same people but boy, I miss the AIX and Solaris troubleshooting tools.

At my previous clients site, they have some old Sun servers that they are upgrading to M5000’s. The hardware they evaluated (and they did very rigorous testing of the hardware) was a T5240, T5440 and M5000. The M5000 was configured with 4 processors not the full 8 that are possible, which is the subject of my post. When choosing the M-series server they decided to go with the M5000 because it would have 2 free slots allowing them to add memory or processors later. So they are trying to protect themselves against a CPU utilization problem down the road by having slots to put additional capacity in. I’ve been down this road a few times myself. I bought the 880s and 890s with 4 procs just in case we needed the other 4 down the road. Unfortunately most of the time I never needed those slots. and wasted the rack space, power and cooling. In my current clients case they should probably go with the M4000 instead.

On a list price comparison over five years you get:

M4000 $66,380 M5000 $81,880

Maintenance (numbers are swags for platinum pre-paid for 3 years):

M4000 $17,000 M5000 $22,000

I’m typically a Veritas user, so that adds complexity. Last time I looked (a year and a half ago) the M4000 was a tier E and the M5000 was a tier H. So SF for oracle for both would be:

M4000 $4,000 M5000 $9,000

Veritas Maintenance would be somewhere around (swag, 3 years):

M4000 $2,400 M5000 $2,700

The rest a bit of a wash, and brings us to:

M4000 $89,760 M5000 $115,580.

So the M5000 which has two advantages: 4 internal drives (no value unless you’re partitioning) and 2 expansion slots (potential future value) has a 28% greater price premium over the M400. The reason engineers like myself make choices like this the unpredictable future. Often times when I’m asked to spec out hardware for an application I’m given initial requirements like 12,000 total users, 300 users concurrently. And if I’m lucky some information about the resource utilization associated with each user session. Most the time it’s a shot in the dark however, and have to dig around for similar usage profiles via google and try to work that into my sizing model. But that’s relatively straight forward. There’s some art and finesse to it but at the end of the day it usually comes down to a derivative formula of X sessions * Y-Mb-per-Session + overhead + wiggle-room = ZGb of memory. Same kind of thing for CPU and I/O. Where it gets hard is when you have to forecast the life of the machine. You’re forced to try and pick a machine that will meet the needs of not only year one but years two through four or five as well. When we ask the customer what their growth rate is they’ll usually shrug and give a non-answer. Or they’ll give you an answer that’s based directly on other non-knowable facts like “our user base will increase at the same percentage as our market share”. Great. Thanks for that. It’s very tempting to just go out and buy the top of the line server to ensure we never get a resource problem. Buy a tour bus when all we need is a passenger van. But when they see the sticker price of that tour bus, we’re usually back to the drawing board. That’s what makes machines like the M5000 or the 890 it replaced so appealing. It has room for an extra row of seats in case the number of passengers we need increases drastically. Unfortunately you have to pay extra fuel costs to hall that extra space around (maintenances) and there’s the up-front acquisition costs as well.

It’s all about going back to the well. The reason we over-build or infrastructure this way is because of the difficulty in going back to the well for additional funding. In my work in the non-profit space there’s a real risk of that well being dry as well. For example, I could buy the M4000 and then if I have problems in year 2 or 3 I would then do a forklift upgrade to the M5000 (swap the boot disks and away I go). Easy stuff, except I have to actually buy that M5000. Which comes with lots of questions: Why didn’t you buy an M5000 from the start? Why were your forecasts wrong? Where do you think we can come up with that kind of money? Collective amnesia will shift all the blame to the people who spec’ed out the system. Blame rolls down hill. It picks up mass and speed as it rolls and engineers are usually at the bottom of the hill with the operations folks (often one and the same). So we buy machines that have that ‘extra reserve’ built in. In high end servers you can usually turn on the additional capacity by purchasing a license key. But in the mid-to-low end range we’re only offered machines with expandability. So if our forecast is off or the conditions change, we’re able to bring a lower incremental cost to the table to gain additional performance and capacity. Unfortunately for me however, I have rarely needed that expanded capacity. I can only remember two examples one success, were we added two boards (4 CPUs + memory) to an 890 and one case where they didn’t make the versions of the board we currently had in the server which meant we would have had to replace all the boards which would have cost almost as much as replacing the server outright.

I used to be a ‘keep something in reserve’ kind of engineer. Be able to put the rabbit out of the hat to meet the increased demand that we didn’t know was coming. Basically pull of a Montgomery Scott to save the day. By doing so however, I have enabled the behavior that has gotten me here in the first place. By not purchasing the equipment the requirements suggest and adding some reserve “just in case” the cycle repeats itself. Now I’m not going to purchase the minimum needed to meet the requirements given to me (however flawed they may be), but I am going to start putting the decision back on the requesters and have them make the choice. In writing. With as much concurrence as can be achieved from the project team as a whole. So if I were to travel back in time to the period before the aforementioned M5000s were purchased. I would offer the M4000s instead. I would tell them you save X dollars up front. Your downside risk is you may have to replace this server if your usage or growth models are wrong. And, perhaps most importantly, get documented concurrence from the stake holders.

This has turned into a much longer post than I had originally intended… phew. Now onto my next client/project.

According to a story in the Register, sun’s next gen server chip “Rock” has been killed. The corresponding servers are also dead. The Sparc T series is still alive and generating revenue and Fujitsu is still pumping out Sparc64, so the future isn’t dire, but I think it’s fairly clear that SPARC as a chipset just went on the endangered species list. On the other hand, like they speculate in the articles, maybe it wasn’t going to work technically or maybe they’re cleaning house before the take over. Either way, the ‘next generation’ sparc is no more.

Great blog post today on The O’Reilly Radar. It graphs the number of online job postings month by month for the last four years. I won’t steal the pictures here, you’ll have to click the link to see them, but it confirms what I’ve been seeing while doing my job hunting. It looks particularly bad for the folks back home in Minnesota. A slight downward trend, with a slight uptick now. Although I have a feeling it will ramp up a bit more for June and July as more government spending trickles out of the stimulus bill. Personally I went the month of March and April with very little job oportunties. Late April and May it picked up slightly (and I landed a short term contract). Now here in June, no fewer than 5 promising oportunites have come my way. It figures, as soon as I take a long term job (W2, not 1099 or c2c) that other options come out of the woodwork. Extensions and increased hours at me current contract. A data center move with CSC (love those, I’m good at it and it has a set end date). A systems engineering lead with the direct competition of the company I’ll be working for on Monday (slightly more senior role to boot… DOH!). A data center manager job… SAN/VMWare engineer… All promising, but I won’t be pursuing any of them. I’ve placed my bets and am going to ride it out for a minimum of 6 months. More likely a year or more. My strategy/thinking there is an idea for another post.

Anyway, good luck out there…

I love unhandled errors from websites. Today, while submitting my timesheet for the day I received the following:

Microsoft OLE DB Provider for SQL Server error ‘80040e14′

Incorrect syntax near the keyword ‘And’.

/EZM/traddfreets.asp, line 752

It tells me that I’ll always have a job in the future. I waited a few minutes and then tried to re-submit my time and got:

Microsoft OLE DB Provider for SQL Server error ‘80040e21′

Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.

/EZM/traddfreets.asp, line 563

These, in particular seems ‘frog march’ worthy. What this tells me is they have an error in production and are trying to fix it IN PRODUCTION! That’s a No-No in my book. I understand the desire to return to service, but what if your ‘fix’ inserts bad data? What if it corrupts? What if it gives access to protected data?

And worse still, The only thing I get is the SQL error. I get no ’sorry’ page, or link to go back to the time sheet, no ‘handling’ of the situation. It’s easy to see how this happens. All to often the focus is on the postitive-outcome side of things. Given A then do B, C and D and then return E. There’s usually error checking along the way, which is half of the negative-outcome side of things. I often see errors and stack tracing output, some of if boiler plate from the underlying components like the ones above, others written by the website folks in question. It reminds me of the Seinfeld episode where Jerry is trying to pick up his rental car and there’s no car available (paraphrasing). “But I have a reservation for a car.” “I know what a reservation is sir.” “I don’t think you do, because if you did, I’d have a car right now. Anyone can take a reservation. It’s the holding of the reservation…that’s really the most important part!” (this bit gets used a lot by a lot of people judging by the google hits). In this case, they know how to ‘Throw’ the exception. It’s the catching, the catching is the most important part.

Oh well, the silver lining is job security.

I have a few domains for my personal and professional use. When I first started my personal website/domain I took what ever came from the hosting company. In my case, www.powweb.com is the provider in question. I’ve been generally happy with Powweb, but I also have straight forward run-of-the-mill needs too. The only thing that was a problem for me was email. I needed to be able to read my mail from multiple machines. Your typical provider only offers POP3 access and webmail. Webmail just doesn’t work for me and multiple mail clients with POP3 is problematic. My initial solution was to use gmail to retrieve the mail via POP3. Then via GMAIL I can use their advanced webmail client and desktop clients via IMAP. I probably would have gone with Yahoo Mail had they offered IMAP for free. Don’t know why, but I like Yahoo’s webmail better for some reason.   

Gmail isn’t with out it’s problems though. When you send email via Gmail from your @gmail.com account it has a funny header that some MTA’s don’t like to honor. So you’ll email will have a header lines:

Sender: user@gmail.com

Return-Path: <user@gmail.com>

but a from line of:

From: Richard Whiffen <me@whiffen.org>

In the past it used to say

From: user@gmail.com on behalf of user@whiffen.org

which was even worse. Anyway, some older mail apps reply to the @gmail.com. This becomes a problem when the reply is to a large group of people. The threads get fragmented because some people are mailing to @gmail.com some are @whiffen.org. Doesn’t happen often, but enough to be a bother.   

The fix is to sign up for google apps. If you have less than 50 mail boxes (not aliases, actual email boxes) the free edition is quite powerful. When you sign up you can either buy a domain via Google, or you can use a domain you already own. I think it’d likely be cheaper via someone other than google but your milage may vary. I already had my domain so I signed up and via some fairly simple steps was able to point my MX records from powweb.com to google.com. So now I can log and get a gmail interface to my @whiffen.org. Google’s mail infrastructure and spam filtering is far more robust than my hosting provider so I have had a noticiable drop in spam since moving over. I also have an online calendar and google docs @whiffen.org as well. So now when I send email from @whiffen.org there’s no funky headers or other issues like that. It does mean however now I have two mail boxes @gmail.com (which I only use to subscribe to listservs) and @whiffen.org (and @rwhiffen.com but that gets no traffic at all). In Mail.app and Outlook it’s trivial to manage. It is a bit trickier from the web interface. I basically have to log in more than once.

Where it gets fun is my G1. I have a T-Mobile G1 the “google phone” if you will. When I bought it I signed in with my @gmail.com account and when I’d click the nice red gmail envelope I’d get my @gmail.com mail and my @whiffen.org mail. But sending mail as @whiffen.org wasn’t possible on the phone with out extra work. The simplest path is to set up the Mail app within the phone (it’s separate from the gmail app). It will allow you to connect to a pop3 or IMAP host. But with the google apps setup, I was able to factory reset my phone and instead of using @gmail.com I used @whiffen.org and it worked like a charm. Now I have a single interface to my @whiffen.org email via the phone, via the web and via my Macs. What’s more, my calendar and calendar invites are now @whiffen.org.

The Google apps setup also comes with Google docs, which I’m using to co-write some documentation currently, very hand tool. It has fairly flexible version control and permissions structure. It can do fairly robust word processing and spreadsheets. I find the spreadsheet navigation a bit clumsy at times do the the web based nature of it. Data entry isn’t as smooth as it is with a local application. I haven’t tried the google gears feature for offline editing yet. But for the basics it’s pretty good. I essentially use it to rough in the documents and the finish them up in NeoOffice or in MS office via parallels.

A hidden gem, I feel, is google sites. Google Sites is very similar to Microsoft’s SharePoint. You have less widgets and flexibility perhaps but you do have a lot of base features. You can make a file cabinet page for simple file storage, retrieval and versioning. There’s a dashboard template that lets you add google gadgets to the page, like weather, docs, excel sheets, movies, but is generally intended to give you a portal-like view into your other site pages. There’s also a announcement template and list template. All together it’s easy to see turning Google Sites into a small company portal for sharing information, which is what I believe it’s intended use is. Although I do find it ironic that it’s not tied into google docs. When you add things to your ‘file cabinet’ page, you have to find the URL’s to your Google docs via the Google docs page and paste them in as a web link. You can’t select them from a list. I suspect this will be improved over time, but I was a bit surprised by that lack of integration.

What will really make Google Apps interesting is when they get integrated into Android. Then your phone will be tied into this nexus as well giving you a lot of power from a phone. I’d love to be able to at least read my docs on my G1. That’s one area that is sorely lacking in the current G1 and the forthcoming 1.5 version coming ‘any day now’. It’ll come some day, but probably not for a while. I think they want to get things like Adobe Flash working first.

If I ever decided to start a small business I’m definitely going to use Google Apps instead of running an email server and share point server. Especially when the first 50 email users are free. I do find it a bit odd that the ‘premier’ edition is $50/user/year, which for what you get, isn’t too steep. That $50 gets you 25Gb of mail, a host of extra security options, a 99.9% uptime SLA, and more support. For a lot of small companies, especially the 10 or less people kind, it would be tough to justify the $50 vs free. The security options might make it work while, but other than that, why would you do it? I’d be curious to know how many freeloaders like myself are out there vs the less that 50 user paying customers there are.

I’ve looked at the ‘live.com’ offerings from Microsoft, and I was very disappointed. It’s far to confusing initially. They’re also trying to be everything web 2.0 all rolled into one. While the idea is reasonable, as with a lot of things Microsoft, the execution is poor. Once I figured out what they were trying to do it made a twisted kind of sense to me but it was still cluttered and confusing. They forgot one of simple things about the web 2.0 experience. Most things are separate by default and you have to choose to join them. Not with windows live, they linked and cross linked everything. I looked at it, acknowledged, and moved on.

Anyway, if you have your own domain I’d strongly suggest you give google Apps a try. It’s great email hosting if nothing else. It’s fast and free. It has extras that appeal to small businesses or groups. If you’re one of the lucky ones who has a grand central account (now called google voice, apparently), you also have a central phone number for your business, again for free.