Software, technology, sysadmin war stories, and more. Feed
Friday, January 25, 2013

When "I care" meets "who cares"

What does it mean when your customers love you, but some of your teammates think you're nothing but a mean old bitch?

In my former role as a pager monkey for a production service, I still had a fair number of customers to support. The difference was that they were fellow employees of the same larger company instead of being random people on the outside who had chosen to use us for service. The system I helped run was the sort of thing which was non-trivial to duplicate, and so teams usually came to us rather than trying to run their own system.

Most of the client teams had project managers, and some of them were more visible than others. One client team in particular handled a nontrivial amount of Real Money traffic and had a PM who really cared about keeping on top of things. They would periodically shift their traffic around depending on countless variables, and would need to know that we would be available to service those requests.

They were pretty technically demanding of us. Their requests needed to be serviced in a fraction of the time most clients would require. It's like the difference between reading from disk vs. reading from memory. Once you're down to caring about things at that level, "little things" like the speed of light also become an issue. In short, our services and their services had to run fairly close together -- geographically -- or we would fail to meet their requirements.

This is detail-oriented stuff, and their PM stayed on top of it. This meant she had to ask a bunch of questions to know exactly where we were, what we were doing, and where we were going next. I had no problem with this, and welcomed the opportunity to share such information, but more than a couple of my teammates didn't have that same sentiment. I used to hear audible grumbling every time she departed from our loft.

One time, from what I could overhear, it seemed like she wasn't getting the kind of data she wanted. I reached out via e-mail to see what was going on. It turned out that one of her clients (one of the biggest internal customers there was in the company) was about to make a large move. They had some kind of maintenance event coming up and were going to start sending her service a whole bunch of traffic in a new part of the world.

This was going to happen in a few days, and if her service (and mine, by extension) were not ready, we would start dropping requests, and that had a measurable impact on the entire company's bottom line. Obviously, this was unacceptable, and something had to be done.

She came up to my desk and we started looking at the config in that region. I did some of the math and found out that no, we weren't ready. We didn't have nearly enough instances of our frontend servers to handle the traffic her service would start sending in that region. While she was still there, I did the *clickity-click* config file mangling to bring things up to the required level and fired off my patch for a code review. It's a small gesture, but it proved that yes, we were "on it", and she need not worry about it getting lost.

To me, this was just plain old customer service. She needed to know that things were ready, and liked understanding the finer points of our load balancing and query handling to see how it really worked. I guess the grumblers on my team really didn't like that kind of interaction, but it seemed totally normal to me. Strange, right?

I was pleasantly surprised when our semi-annual peer review season rolled around and there was a nice comment from this person in my report. How can you not appreciate someone going to trouble to write this about you?

She always takes the time to explain in detail what is going on and how I could check for myself in the future should the same question arise for another region. For example, she walked me through the process of seeing which [TECH] a [TECH] front end prefers to talk to, and described what happened if a preferred [TECH] suddenly goes offline. I really appreciate understanding how things work as it helps me decide which [big internal customer] plans are and are not going to adversely affect [my service]/[her service] performance.

How awesome is that? I didn't do anything special, really, but I got a great bit of feedback for it in a venue that would be beneficial for future raises, or promotions, or whatever. It was a nice change from some of the stuff people used to put in these things.

Want to see what someone else wrote during the same review period? Oh, okay, if you insist. Here, get a load of this:

Rachel's 'prickliness' continues to be one of her greatest challenges. Whether only perception or not, she comes across as terse and easily angered or frustrated. While she still shows remarkable skill in keeping professional even when in this state it makes it difficult to enjoy working for her during these times.

Prickliness. I wonder if that was originally written as "bitchiness".

That little excerpt is courtesy of one of the people who would grumble out loud when the above PM person would leave the room. I wonder if he would have said the same things about her, given the chance.

This isn't a simple guys-vs-girls thing, either. In that same period of time, another one of my customers from a totally different team happened to show up on my review. This person's product was far away from the company "money spigot", but it didn't matter to me. He wrote something wonderful about the support I provided, too.

Rachel always fields my question with respect (even the stupid questions). It really puts me at ease knowing that Rachel is on the other side of the system and I can go to her with questions, concerns and panics.

It's true that I saved this second person's bacon a couple of times when one problem or another threw some important user's data in the wood chipper and I figured out how to restore it somehow, but it wasn't all about firefighting. They actually managed to deliver several 100% availability quarters, and they wouldn't have been able to do that if we weren't also delivering that level of service.

Here's why this matters to me. Imagine if you have a kid, and she goes out into the world and does something that's totally socially unacceptable. This reflects upon you as a parent! How could you bring up a kid to operate that way?

Well, when it comes to my service, it's much the same thing. If my service misbehaves and lets someone down, it's almost the same as if I personally let them down. The service is an extension of myself, and if it looks sloppy and broken, then it serves to describe me the same way.

That's just the way I feel about these things. It's a sense of ownership which just won't go away. Somehow, I don't think this feeling is universal. To some people, it's "just a job", or, even worse, just a paycheck. Here's an actual exchange:

Me: Do you ever think we keep fixing the same problems in this industry?

Him: Yes.

Me: But um, what's the point?

Him: Well, this may seem shallow, but these problems are buying me a new BMW.

That's me talking with the guy who used the word "prickliness" earlier.

If it's "just a job", let me know, okay? I want to get away from you.