books & summer vacation

I set a goal for myself during the past nineteen days while we were all on vacation: Read a bunch of books.

And while I didn’t hit my goal of 9, which, let’s be honest, was pretty aggressive, I did finish four books and am half way through Hemmingway.

Here’s a rundown of what I did read:

“Daring Greatly: How the Courage to Be Vulnerable Transforms the Way We Live, Love, Parent and Lead“, Brene Brown

“Sometimes the bravest and most important thing you can do is just show up.”

Life altering might be an over exaggeration but I was moved enough that I gifted copies of this book to several people.

I will continually ask myself, “What’s worth doing even if I fail?”

I will continually ask for the “courage to show up and let myself be seen.”

“Bossypants“, Tina Fey

This has been on my list for some time and after my wife finished reading Sheryl Sandberg’s “Lean In“, we co-read Fey’s book.

Super simple read. I’ll leave with two highlights:

  1. “In most cases being a good boss means hiring talented people and then getting out of their way.”
  2. When dealing with people who are critical, ask yourself the following question: “Is this person in between me and what I want to do?” If the answer is no, ignore it and move on.
“The Five Dysfunctions of a Team: A Leadership Fable“, Patrick M. Lencioni

This was as easy a read as “The Phoenix Project” was. Told in a narrative format, reading about how teams work or don’t work.

I fundamentally believe that at the root of everything must be trust. Trust and authenticity.  I was taken by this:

“trust is the confidence among team members that their peers’ intentions are good, and that there is no reason to be protective or careful around the group.”

and,

“By building trust, a team makes conflict possible because team members do not hesitate to engage in passionate and sometimes emotional debate, knowing that they will not be punished for saying something that might otherwise be interpreted as destructive or critical.”

“How Children Succeed: Grit, Curiosity, and the Hidden Power of Character“, Paul Tough

Through Year Up the biggest skills I see as important to tomorrow are the ability to learn, the ability to be more than you are today.

Paul calls this Grit.

“The problem, as Randolph has realized, is that the best way for a young person to build character is for him to attempt something where there is a real and serious possibility of failure.”

“A Farewell to Arms“, Earnest Hemmingway

I’m not actually done with this but since I read it back in high school, I’ll count it.

This was my favorite book in high school, even better than Thomas Pynchon’s “Crying of Lot 49” but I can’t remember exactly why.

Figured a second read might be a good idea.

on leaving mozilla

Mozilla IT 2012

Mozilla IT 2012

Last Wednesday, July 31, I left Mozilla as a full time paid-staff.

Seven years at Mozilla has fundamentally changed how I operate. It’s redefined my values and helped define my personal mission & purpose.

The impact I had on Mozilla and on the Mozilla Project extended far outside Mozilla IT Operations and I hope to continue to be connected to the Mozilla Project (but more on that in another post).

For now I’ll simply share the note I sent to Everyone@Mozilla:

Adios, au revoir, sayonara

On March 15, 2006 I quietly joined Mozilla. Seven years and four months later, I quietly take my leave.  July 31 will be my last day here.

I had this moment of clarity awhile ago when I realized I have an awesome set of leaders under me and it was time, as Mitchell has said, to let them step into new roles and for me to step out.

To my Mozilla IT:

I have enjoyed being there for you.

I have enjoyed seeing you grow.

I have enjoyed seeing you handle some crazy, complicated challenges.

I will miss the camaraderie we developed and shared with each other.

I will miss the late nights working with you.

You always worked with me, never for me. And as much as I tried to help you, you helped me.

For this, and much much more, I thank you.

To my friends,

I made friends with people I worked with, people who are friends first and co-workers second.  I will miss seeing you every day. This is probably the hardest realization.

You have helped me grow and become who I am today.

To All@Mozilla,

In many ways, you, and this organization, have made an impact on me that cannot be undone.

My thoughts and actions have been shaped by what we have collectively been accomplishing and what we have set out to accomplish.

I could write much much more, of course so I’ll just leave with this:

“Parting is such sweet sorrow that I shall say goodnight till it be morrow.”

failure

“I’m not afraid to fall; it means i climbed up high
to fall is not to fail; you fail when you don’t try”
~ Superchick “Get Up”

 In the middle of a long text message conversation on my way home on BART I made the comment, “Half the stuff I’m working on now I’ve never ever done. I picked it because I’m most likely to fail at it!”

I blurted that out but was instantly reminded of something @MitchellBaker said (and I’m paraphrasing) ~ “If it makes you feel uncomfortable, you should probably do it.”

Stepping out

Over the past year I’ve spent a lot of time stepping outside my comfort zone. Way outside.

Each time is a total gamble.  Each time I risk failure, disgrace, embarrassment.

Because not doing so is absolute failure.

I fell. It hurt.

I understand that failure itself isn’t Failure.  “Failure is required” as @cookflix said.

It’s really no different than a child learning to walk.  You fall. You get up. You take a step and you fall again. You get up.  You keep at it, you keep at that which is important.

As a leader

Over the past year I’ve found myself developing my leadership style, my brand, my way of leading.  I’m honing in on who I am.  Who @mrz is.

I try very hard to empower others, to encourage, to aim to surround myself with people who can do a better job than I can.  People who can stand on my shoulders and do more than I could possibly do. In many ways, I try to delegate as much as I can, to hand off as much as I can.

And I’m more comfortable taking gambles. I know what matters to me (mission, empowerment). I know who matters to me (friends). And within those contexts I feel safe.

I take risks.  I put myself out there.  I iterate.  In the words of @msurman, “Fuck it, ship it.”

You should too.

“Something comes from nothing if you’re willing to believe.”

~ Pillar, “Everything”

7 years @ Mozilla

A little more than 7 years ago today, I was at a park in Aliso Viejo with my then 2-year old son when @dan_portillo called. Mozilla was looking for a Network Engineer.

A little more than 7 years ago today I was interviewing in Mountain View. I carried a cell phone that could only make phone calls.

7 years ago today I walked into Mozilla’s Landings office and configured my own Thinkpad laptop by myself (we didn’t have Desktop Support).  

Two weeks later Mozilla’s company-wide All Hands fit in the back of Tied House. Two months later I moved my wife and two babies from Orange County to Northern California.

Along the way I bought a house and called this home.

7 years.

7 Years of Change

I’ve gone through 3 CEOs & 3 bosses. My team’s grown from 3 to 80. I manage an IT Operations team. I’m learning how to lead, how to lean into my strengths.

I’ve learned to delegate, to trust, to let go. And I’ve been surprised and impressed at every step by the caliber of people who chose to surround me.

I now have co-workers friends whose first names I don’t know and who I’ve only met in person less than a handful of times.

Before Mozilla I had only traveled internationally once. For my honeymoon.

I’ve been to Argentina (pictures) & China twice. Stepped foot in Peru. Been to Whistler twice. I’ve seen a bear eat out of a trash can and stood on top of a mountain in July. While it snowed.

I’ve been to Berlin, Switzerland, Amsterdam, Vancouver. Lost count how many times I’ve been to Toronto, to London, to Paris. 

I traveled to Nice and stayed with a fellow Mozillian. Half my twitter stream is in Spanish. My arguably best friend lives 8500 miles away.

Along the way I lost my passport.

7 years ago my first smartphone was a Palm Treo running Windows. I now carry three phones & a tablet and saw the future I want last year as Mozilla shifted to Firefox OS. I have — and want — the web in my pocket.

7 Years of Personal Change

My 9 month old daughter is now 7 (or 13, it’s really hard to tell). My son knows how to find my name in about:credits.

I’m deeply focused on being an advocate for Mozilla Webmakers. I care about eliminating the opportunity gap through Year Up. I’ve learned that a mission with purpose means more than profit.

7 years ago I was a shy introvert. I still am but you probably wouldn’t know it.  7 years ago I was a Windows user.

I’ve become a San Francisco Giants fan & saw my team win the World Series. Twice.

In the past 7 years I’ve made friends. 

Along the way I learned that without music life would be a waste. 

Celebrating 7 Years

I don’t want any gold watch.

I want to do the same thing I’ve done every day for the past 7 years. I’m going to go to work.  

I’m going to go and work with people who make Mozilla not just a job, who have made this my home for 7 years, and who are as passionate about Mozilla & the Open Web as I am.

barcelona. friends.

Last night in Barcelona and I find myself in El Argentino listening to a single saxophone, my second most favorite instrument after Spanish guitar.

Quite a spectacular way to end. It bookmarks two of the most amazing things I’ve ever done with Mozilla ~ one in Buenos Aires and one in Barcelona.  And magically I’ve been to both cities twice and each trip opened my eyes to new ways.

Feels fitting to sit in this restaurant. In this town.

This trip was unique for a number of reasons but it was the friendships that grew that mattered most to me.

I just re-read a text message I sent to someone before I left.

Would love nothing more than to hang out and get to know you better.

Friendship is an odd thing to me.  Perhaps it’s just something I reflect on more as I approach my 40th birthday. 

With the exception of the small circle of college friends whom I still keep in occasional contact with, most of the friends I’ve had in life disappear as life changes, as jobs change.

I want a change.  I spend an inordinate amount of time with friends I work with, the ones who make my job a fun place to go to everyday.  It is these people who I want to keep as friends outside of work.

On this trip I changed how I related to them.  It wasn’t really hard.  They probably don’t know it but I really like them.  They fascinate me, they interest me.  Each has differing shared experiences with me that no one else can relate to.  I moved outside my comfort zone.  I let them know I cared about each one.

Sometimes that meant waiting behind to escort a friend home.  Sometimes it meant skipping on plan A so a friend wouldn’t be left alone.

And then this thing happened.  New bonds were formed, barriers removed.  It felt entirely right to share meals with each other, to order a couple dishes and simply share (tapas certainly helps that but it was more than the style of food.). Felt good.

You saved a seat for me when I was running late.  I wanted to wait for you when you were running late.  You waited for me, you included me. You shared with me, I shared with you.  We explored together.

Is this what friendship is? 

To each of you who made my trip something I couldn’t have imagined otherwise, I love you all. Thank You.

i think

[This is a little old, probably written sometime in 1993.]

I think of the stars in the sky
   how lonely they must be,
   so far apart from each other.

I think of a candle without a flame,
   missing that which makes it whole.

I think of the winter trees with their leaves all gone,
  how sad they must feel.

I think about the tumbleweed,
  blowing across the open land.
  Lost, wandering about, not knowing where to go.

And then I think about you
  and suddenly,

        the stars aren’t so lonely,
        the trees aren’t so sad,
        the candle shines brightly,
        and the tumbleweed…
        the tumbleweed isn’t so lost.

one planet. one people.

I wound up taking most of the day off from work yesterday to drive 200 miles to help a Friend.  It was a long and exhausting day.  This is a person who ten months ago I didn’t even know, let alone care about.

But this isn’t a post on what I did.

It’s that I was struck by a number of people and things that came together to help my Friend.  A relatively large number of selfless people.  Enough that thinking about it makes my eyes water.  Enough that even writing this post causes me to choke up.

I was struck that amongst all those who I interacted with to help my Friend, not a single person did so with any expectations or desires of anything in return.  We helped, simply, because it is what you do.

  • There was the person at work who made sure I was going to drive my Friend.  She made it clear that that not helping would have been a personal disappointment.
  • There was the person who moved mountains in what felt like record time to make sure my Friend had certain resources.
  • There was my other friend who did me a favor to help my Friend, who pulled in his own circle to help.

Lastly, my children.  My children have meet my Friend exactly twice and once was last night.  Yet they exhibit all the passion and care for humankind a parent could only wish for.

  • There was my daughter who made it absolutely clear that I needed to make sure my Friend knew to call me whenever, should my Friend need help.  My seven year old daughter.
  • There was my son who checks in with me daily asking how my Friend is doing, who reminds me we have a spare room should my Friend need a place to stay. My nine year old son.

There is that adage that it takes a village to raise a child.  The truth in that felt real yesterday.  There was this sense all day that here we all are, many voices, one planet, one people.

39.

I turned 39 this past Sunday. 

My daughter and I share the same birthday (as does @selenagomez!). Over the years this has turned into a fun game of arguing over whose birthday it is.  And over these years my priorities of what I want on my birthday have changed.

  1. Farrell’s Ice Cream Parlor.

    When I was younger I remember going here for my birthday.  All the pomp and circumstance! This place more or less disappeared.  When I saw they had re-opened a few in Southern California I knew what we were going on my LIZ’s birthday.

    I couldn’t have been happier when the band came out, drums and horns blaring and LIZ on her chair. She’s never seen anything like this. She had this crazy big smile on her face as she soaked up being the center of attention (for literally the entire restaurant).

    In that very instant I saw the young woman she will eventually become. Full of confidence and poise.

  2. My son took me to lunch.

    Those who know me know I enjoy a great culinary experience. I like tasting new foods. I love the presentation.

    When WNZ asked me what I wanted for my birthday, I told him I only wanted him to take me out for lunch (well all four of us). I figured I’d give him some cash and let him treat. The more we talked about it the more I could tell he was getting excited about doing it.

    He took me out to lunch at Opah Restaurant, a place my wife and I used to visit when we lived in the area. I kept trying to give him money but he’d have none of that.

    “Do you need some money?”

    “No Dad, I’ve got this.”

    And in that instant, I saw the young man that he will eventually become. Full of independence and pride.

I may not be a man with a lot of material wealth but nothing could have made me happier than those two things this weekend.

Zimbra & Mozilla email, 4 months later

Four months ago was a very tough time in Operations. We suffered a catastrophic disk array failure on Mozilla’s mail server (I blogged about it too). A series of mistakes kept email offline for two days. This was the worst I’ve ever felt, both professionally and personally.

Fast forward to today. So. Much. Better.

We learned. We researched. We re-organized ourselves. Much like The Six Million Dollar Man, we rebuilt it better.

justdave posted his account, “Re-imagining Zimbra email at Mozilla” but I wanted to add my own color.

Background

During my interview at Mozilla in 2006, I was asked a bunch of questions about Zimbra. First I had heard of it. By the time I started I had learned quite a bit about Zimbra. Back in 2006, mozilla.com email was hosted externally and we began the process of moving email back in house. The company hosting email couldn’t provide SSL and wasn’t doing all the groupware things we needed.

Post Zimbra-gate (December)

I was mentally done with email. I looked at simply outsourcing. I looked at hosted Zimbra, hosted Exchange, hosted whatever.

We Mozillians, we’re a unique group.

  • We want to use the IMAP client of our choice. Some of us just want to use the web interface. Others prefer Microsoft Outlook. Or Thunderbird, or Mail.app or Postbox or mutt or pine or Sparrow or …
  • Calendaring is just as complex.
  • We need to support a wide number of mobile devices – iOS, Android, Blackberry, devices that support Microsoft’s ActiveSync – with both email and calendaring.
  • Some use Zimbra’s document sharing/storage
  • We need something that supports IMAP, ActiveSync, CalDAV, CardDAV.

We looked at what others at our scale and beyond our scale use for email. Oracle uses Zimbra. Comcast uses Zimbra. At. Scale.

We talked to others hosting their corporate email with Google Apps (and their 15-person staff managing their Google Apps mail!). We learned that deploying Exchange requires a move from OpenLDAP to Active Directory and a particular skill set that we don’t have in house.

Moving Forward

This incident highlighted the need to have a team focused on infrastructure. Our primary focus (and priorities) always tend to lean towards various Mozilla web properties or developer services.

So we did two things –

  1. Broke up a fairly flat Operations group and created an Infrastructure Operations team (and a couple others) to focus on services like email & LDAP, to name a few.
  2. Built a new environment for services that, when break, cause work stoppage, cause a line to form behind my desk. This Hyper Critical Infrastructure, or HCI, is isolated from the rest of the production environment, has different change control processes and is meant to hit as many “9s” as we can hit. It’s a very different way of planning than we had done in the past.This technology stack uses more corporate/enterprise technology than we’re used to using at Mozilla.

HCI Today

HCI straddles two high density, (~15kW) racks. It’s only relation to the rest of Mozilla production network is two 10GbE fiber drops from the network core.

HCI has it’s own Juniper SRX 1440 firewalls. Its own Juniper EX4500 switching. Its own NetApp FAS3270. Its own 5 node VMware ESX cluster, each machine having 2x 6-core Xeons & 192GB RAM.

In a couple months, services here will be replicated to SCL3 using various NetApp & VMware technologies.

We had planned to have HCI in production by the end of February but no one wanted to rush this (plus someone decided to have a baby).

Instead we slipped that to the last week of March and I’m glad we did. We consulted with Zimbra and others. We sent Desktop & InfraOps to training. We tuned and fine tuned.

Zimbra Today

We have mailboxes spread across seven mailbox servers and understand the metrics we’ll use to determine when to add more mailbox servers.

We migrated 1002 mailboxes from San Jose to Phoenix without anyone noticing, without any user impact, in just a couple days. In fact, we didn’t mention it until we were done.

We have instrumentation and trending and alerting on everything we could think of.

What’s next?

All is for naught without learning. We learned a lot and we’ve changed how we operate as a team.

Once bitten, twice shy.

A Test

[This is a short story written in 1991 while at UIUC.  Lyric quotes are from some Enya song, name escapes me now.]

He stepped out into the cold, glad to be through and glad to be out.  The cold winter wind slapped him in the face as if scolding.  He hadn’t really bundled up too well – just wanted to leave.  He stood there now pulling together his jacket, fastening the zipper and throwing his bag over his shoulder.  Sure was cold tonight.

“Damn…” he though to himself.  “What the hell’s happening to me?”  The cold wind stopped and stood still and for a brief moment, there was silence around him.  He reached into his bag, felt around, and produced his walkman, perhaps the only thing he valued tonight.  “Got a long walk back.”

Pretty cold tonight.  Pretty dark too, and quiet, as if everyone knew and wanted to leave him alone.  Everyone else was probably still in there.  There were perhaps many going on tonight and many people were probably still finishing up.  He didn’t care – just glad to be out of there.  “No use in sitting around here any more.” he has said to himself as he had grabbed his bag to leave.

It was cold.  Cold and dark.  The wind had picked up.  He had felt alone before, but tonight he felt really alone – and sad.  He had never failed like this before.  “Fuck it – who gives a shit anyways?” he said out loud, breaking the stillness, as he flicked on his walkman.  As if to abuse him, a brief gust of wind blew by.  His ears stung.  He could imagine what they would say, and wanted only to block it out.  He shrugged.  “Life goes on.”  He shrugged again, as if to signify something important, and started back.  He shoved it back into his pocket.

Finding nothing interesting on the radio, he hesitantly freed his hand to brave the cold, and thumbed for the play button.  For the past five months, music had become his one place of refuge.  He could block out anything with the flick of a switch.  If only that worked in real life. ‘…Eurus…’  He still had a ways to go.

It was cold and reminded him of happier days; days gone by delivering papers.  How he was alone and could be himself.  How he would daydream of his other life (how he wished he was there instead of here), and for a short while, all would be okay.  How he would yearn for the house up next with the Christmas lights strung up, smiling at him.  He would pay them a little attention as he stopped to warm his hands in their glow.  ’…if every man is true…’

“How romantic a night!” he thought.  “How I wish I had someone special in my life.” A though which brought a tear to his eye.  Someone to take a walk with on a night like this.  Someone who would care because tonight he didn’t.  He’d never been in love before.  He rounded the corner and could see the lights ahead.

This wasn’t his life, was it?  He wished with all his heart that someone would come and take him away.  Away to the place he belonged.  But that was there.  ‘…etu itu ad astrum…’  He was here.  The wind stopped yet again, and there was the noise of passing cars and people talking.

How warm he felt as he pushed open the door to the builing and headed down the hall.  She saw him quickly, and started towards him.

“Your back early!”

“Yeah, it was a breeze.”