Caitie McCaffrey
  • Years in Tech

    10

  • Current Role

    Tech Lead of Observability, Twitter

  • Place of Origin

  • Interview Date

    February 5, 2016

I started coding in high school when my computer science teacher sought me out and encouraged me to take her class (I was in advanced math).  I fell in love with programming, went to college for it, got into game development, worked on games like Gears of War and Halo while living in Seattle. I moved down to San Francisco last year to become the Tech Lead of the Observability team at Twitter.

Let’s jump right in. Tell me a bit about your early years and where you come from.

I grew up in the Midwest first in Pennsylvania until I was 9 – and then Ohio, and I started programming in high school. During Sophomore year I decided to be lazy and take an extra study hall, but the computer science teacher, Mrs. Petite, hunted me down, along with all the other advanced math students, and said “You should take my class,” and I’m so glad she did.  I took two and a half years of computer science in high school and just absolutely loved it. My first class was in Visual Basic, and I made the video game Frogger as my final project.

At the end of my sophomore year I was very nervous, I knew that my school had AP computer science courses but I wasn’t sure if I would be good at it.  I asked my teacher Mrs. Petite who said, “Of course you would be. Sign up.” So I did. I took AP computer science A junior year in C++ and AP computer science B senior year in Java.  I fell in love with programming in high school, I loved how quickly I could bring my ideas to life, and so I went into college knowing I wanted to major in computer science.

“I fell in love with programming in high school, I loved how quickly I could bring my ideas to life, and so I went into college knowing I wanted to major in computer science.”

How was college for you?

College was great! I always knew from a young age that I wanted to do engineering or math or science based things. I just liked them in addition to a healthy appreciation of the art.  I spent a lot of time writing, reading literature and dabbling painting and pottery. But I had a penchant for the sciences, so when I applied to colleges I focused on schools with computer science departments.

I eventually chose Cornell University in Ithaca, New York which has a really excellent computer science program.  It is a super tough program but it was really fun. I have some best friends from that program that I still very close with even though we live in different cities.

In college I also TA’d a class that was the Intro to Data Structures course in Java.  I also became interested in game design and built a couple video games, via an elective program.  I was also incredibly interested in computer graphics because I liked the idea that I could visualize or see what I was making and create art. I also came to realize that I was really bad at linear algebra, or not really bad but I was never going to be great at it, so then I thought maybe I should pick a different area to focus on.  

During college I interned at bunch at different places including Stanford’s Linear Accelerator Center (SLAC) and Microsoft.

At SLAC I had my first introduction to large scale computing.  They had, at the time, one of the larger academic computing clusters that the scientists working at the accelerator could use to analyze and compute all the data they were collecting.  In order to do this they needed massive compute, and I worked on the team there that provided it.

I also did a more traditional internship after my junior year at Microsoft, in Seattle.  I worked on Windows Live Experiences which was how I got involved with networking and found a passion for building things that socially connect people.  

When choosing a full time job after college I really wanted to work in Video Games.  I wasn’t exactly sure what area of computer science I wanted to focus in so I figured I might as well just build something that I think is fun.  I ended up landing at Microsoft in Microsoft Studios because I got an offer to work on the Gears of War Series, an Xbox Game.    

“When choosing a full time job after college I really wanted to work in Video Games.  I wasn’t exactly sure what area of computer science I wanted to focus in so I figured I might as well just build something that I think is fun.”

So let’s dig into those first experiences. How was working Seattle? How was working in games?

I loved working in games. I’m really glad I did it in my early 20s. I’m also really glad I’m done, or am on a break from the industry at least for awhile, because it is pretty exhausting. The crazy hours you hear about are 100% true, but I learned so much in those first six/seven years I spent in games because you just work all the time. I loved it and I was having a blast so I don’t regret any of it, just sometimes your priorities change away from just work as you get older.

“I loved working in games. I’m really glad I did it in my early 20s. I’m also really glad I’m done, or am on a break from the industry at least for awhile, because it is pretty exhausting.”

So I worked on Gears of War 2 which was a phenomenal opportunity straight out of college that doesn’t happen in the games industry that often. So I got very lucky and I worked my ass off. We were working 60+ hours a week as soon as I started, and I remember sleeping on the floor to finish builds and taking naps as things were compiling. Game builds actually take a really long time because you have to compile the assets down and then package it into something the Xbox knows how to run. There’s this xkcd comic of, “Oh, it’s compiling” and that’s a real joke in the game industry.

I then continued to work in the Gears of War franchise on Gears of War 3.  As we were working on that—an internal studio instead of Microsoft—343 Industries was created to take over the Halo franchise from Bungie, the studio that created Halo.

“We were working 60+ hours a week as soon as I started, and I remember sleeping on the floor to finish builds and taking naps as things were compiling.”

In 2009, I was loaned out to 343 to work on one of the first title applications on the Xbox360, Halo Waypoint.  Originally on the Xbox360, you had this standard dashboard and all you could really do on the console was play games, I don’t know if people remember that, this was a while ago. And then they added the applications like Netflix, Hulu, Youtube, etc…. Halo Waypoint was also one of the first of what we call title applications, that came out.  It was very new, and I jumped on it because I thought that was really exciting.

At the end of the Halo Waypoint project, I learned that 343 Industries had positions open, so I moved over to that studio full time when it was about 40 or 50 people. I was Web Service Developer number 2 that they hired, so that was pretty cool because we got to take over this whole charter of the Halo universe. I was really young, only 23, so this was an amazing opportunity, so I just ran with it.

Halo was really fun, it was this crazy wonderful experience. It was very career-defining for me. It’s definitely the biggest thing I’ve done at this point. We re-wrote all of the Halo services in Azure, which is Microsoft’s cloud compute. We also built a lot of it on top of this Microsoft Research technology called Orleans. The Orleans team was amazing we approached them and said, “Hey, we like your stuff. We would like to build all of the Halo services. Will you work with us?” and they were crazy enough to say, “Yes.”  Then we were crazy enough to think we could do this so we did it and it was amazing. It really led me to love, not only games and creating the social aspects of it, but also introduced me to Distributed Systems.  

What was it like to be a woman in the Game Industry?

There’s obviously a lot of controversy going on about the games industry now. Everyone knows about all the harassment, especially a lot of women are facing in that industry, especially online, and all the horror stories that are coming out. I actually had a very positive experience, at least with my studio. I attribute a lot of that to the fact that Bonnie Ross is the studio General Manager, and she runs all of Halo at Microsoft. And then the executive producer of Halo 4 was Kiki Wolfkill, another really fantastic woman. And so, when I joined the studio, I was literally joining a studio run by women, and it was fantastic. I think it just set the tone. Even on little things like t-shirt orders always included female-sizes even when we were only ordering like 30 shirts, things like that. The studio always felt very inclusive.

Trade shows were a little different. People would come up to me and say, “Oh, you’re not in marketing?” And I’m like, “No dude, I make the game.” And that was always surprising to people.  I used to count the number of times a person said “Really?” I got up to five once.

“I actually had a very positive experience, at least with my studio. I attribute a lot of that to the fact that Bonnie Ross is the studio General Manager, and she runs all of Halo at Microsoft. And then the executive producer of Halo 4 was Kiki Wolfkill, another really fantastic woman. And so, when I joined the studio, I was literally joining a studio run by women, and it was fantastic. I think it just set the tone. Even on little things like t-shirt orders always included female-sizes even when we were only ordering like 30 shirts, things like that. The studio always felt very inclusive. Trade shows were a little different. People would come up to me and say, ‘Oh, you’re not in marketing?’ And I’m like, ‘No dude, I make the game.'”

So you left 343 Industries? What was the impetus?

Some of it was just—I’d been at Microsoft for five, six years, something like that. Problems start looking more of the same and there was just some things I wanted to go explore outside in other parts of the industry. I was also very burnt out. To be perfectly honest, I was working really long hours even after we shipped Halo 4.

Shipping Halo 4 was one of the most magical, sleep-deprived, wonderful experiences of my life.  I loved every minute of it and leaving Halo in 2015 was really hard, but sometimes you get to a point where you want to do or explore other things.

I then did a year at HBO, which started up a studio in Seattle.  Essentially I was working on what became HBO Now.  I stayed there for only a year.  It wasn’t the best culture fit for me.  

So then I made the decision to move to San Francisco.  I knew a lot of people in San Francisco. I was super excited by the tech scene, just because a lot of stuff is happening and it’s interesting. So I started looking around and interviewed with literally everyone [chuckles]. I had a bunch of job offers from some really amazing places but ended up deciding on taking a job at Twitter, and so that’s where I am now.

What excites you in your work now? Is there a thread across all your work, from games to Twitter?  

I really love Distributed Systems.  The space is developing really quickly as applications have to scale to handle larger amounts of data that’s globally distributed.  In addition we now expect services to be always on.  Down time isn’t acceptable.  So there are a lot of really hard problems in this space, and a lot of overlap between research and industry that really excites me.

I also love Twitter as a platform.  Just like games, I’m helping to build social connected experiences that bring people together. I’ve learned a lot on Twitter, and made some really close friends who I first connected with via Twitter.  I also think Twitter does a lot of good in the world, giving people a voice.

“I really love Distributed Systems.  The space is developing really quickly as applications have to scale to handle larger amounts of data that’s globally distributed.  In addition we now expect services to be always on.  Down time isn’t acceptable.  So there are a lot of really hard problems in this space, and a lot of overlap between research and industry that really excites me.”

How did you get into speaking and writing?  How has that experience been for you?

So I kind of got pushed into speaking actually.  I’ve been very fortunate in my career to have a bunch of people who have sponsored me.  So at Microsoft, Clemens Vasters and Scott Hanselman sought me out and encouraged me to come talk on their podcasts.  I also have a friend Kate Matusdaria who made the intro that led to my first conference talk at Surge 2014.  

I also got invited to speak at Ordev 2014 basically because I got into a heated argument on Twitter with Kelly Sommers and some folks from the Node.js community about Orleans.  Afterwards one of the conference organizers reached out to me saying, “We are really interested in learning more about Orleans and saw the conversation on Twitter, would you like to come talk about it.”  So I responded “I guess, sure. That sounds great.”

My first conference talks went over very well, from those came more talk invitations and most recently an article on The Verification of a Distributed System that was published in the Communications of the ACM.  

What’s it been like being a senior-level woman in Tech in Silicon Valley?

It’s definitely hard, and it gets harder as you get more senior I think.  But I’m in tech because all the toys I want to play with are here, and I love the work I do, so I’m here for the long haul.  

Most days it is fine but every now and then there are things that grate on you.  There’s a lot of little things, like getting offered “uni-sex” aka men’s sized shirts.  Or people being surprised when you say I’m a Tech Lead.  Then there are some people who want to see your credentials.  At this point I’m just like, “Do you need a moment to Google me, hold on I’ll wait.”  Honestly the reason I’ve been hustling so hard the past few years, all the talks and articles outside of my day to day job, helps with my visibility/credibility in the industry.  It makes it easier to show up, because some people already know your name.  

“Most days it is fine but every now and then there are things that grate on you.  There’s a lot of little things, like getting offered ‘uni-sex’ aka men’s sized shirts.  Or people being surprised when you say I’m a Tech Lead.  Then there are some people who want to see your credentials.  At this point I’m just like, ‘Do you need a moment to Google me, hold on I’ll wait.'”

Where have you traditionally found your support networks, and where do you find them now?

My family has always been a huge support.  In the tech community I’ve met a lot of friends via IRC and Twitter, I know that sounds super nerdy.  One of my best friends, Ines Sombra, I knew because of Twitter and we met for coffee when I was in the process of moving down to San Francisco.  There is also a group of people I know from IRC channels, where we talked about distributed systems, and since have become friends.  I’m really grateful for all of them because I have a really strong network of women and men who are very supportive, who get it, who have my back, and I think that is really important for staying in the tech industry.  I do not know how I would do it otherwise.  

How do you think your background and family and support networks and life experiences impact the way that you approach your work?

My mother has always had a career, a very strong one, she’s currently a Director at the US Department of Transportation right now, and has been CFO at companies.  She’s always been managing teams and directors of them, and implementing these big projects. You know, just seeing her out there doing that makes me believe I can.  So her career definitely influenced me early on to also want that for myself.   

Have you had other specific role-models in your life? People that have inspired you?

I feel like I owe quite a bit to my high school Computer Science teacher, Caroline Petite.  If she hadn’t encouraged me to take her class, I probably wouldn’t be where I am today.  Her passion for programming and teaching really inspired me.  I still stay in touch with her via Twitter it’s great, she’s fantastic.

Of course Barbara Liskov, one of the greats of distributed systems, and one of only two women to win the Turing Award.  Also Nancy Lynch, she wrote one of the canocical distributed algorithms books, and has done foundational research in the space.  Just seeing them out there excelling in the space is inspiring to me.  I hope I get to meet them one day.

“I owe quite a bit to my high school Computer Science teacher, Caroline Petite.  If she hadn’t encouraged me to take her class, I probably wouldn’t be where I am today.”

Then there are the women I know more personally who inspire me; Camille Fournier, Ines Sombra, Jessie Frazelle, Charity Majors, Jill Wetzler, Elaine Greenberg, all these women in tech inspire in different ways, and I’m lucky to count them as friends.  

What really excites you about tech right now?

So I get really excited about academia and industry partnerships.  I think we are in a very fortunate time where these key partnerships are starting to happen, because I think we’ve hit the sweet spot where the industry actually faces some of the problems that academia is trying to solve in distributed systems.  In the 70s and 80s when some of the canonical distributed systems papers were written people were still running things on one machine, so it wasn’t as applicable to industry at that point.

I have been fortunate in my career to work with some wonderful researchers like The Orleans Team at Microsoft Research.  I also am lucky to get to chat with academics like Peter Bailis and Peter Alvaro and pick their brains, and hopefully partner with them sometime in the future.

So I get really excited about academia and industry partnerships.  I think we are in a very fortunate time where these key partnerships are starting to happen, because I think we’ve hit the sweet spot where the industry actually faces some of the problems that academia is trying to solve in distributed systems.”

What would you like to see change in Silicon Valley?

I think Hero Culture is really harmful.  The idea of this one awesome developer who jumps in and saves the day, or founded the entire company, or wrote an open source library all on their own.  I get that it’s a good story, but I’m always skeptical when I hear them, because I’m like did they really or is this just the myth we are telling?

I also think Hero Culture can be really toxic to teams.  It doesn’t foster a collaborative environment.  It’s not helping other engineers on your team grow, especially the junior engineers.  It also creates single points of failures and bottle-necks in your organizations.

But hey, it’s a good narrative.  Who doesn’t like a good story where the hero comes in and saves the day. It’s a much easier story to package and sell versus the team of complicated individuals that had to band together to do something.  

“I think Hero Culture is really harmful.  The idea of this one awesome developer who jumps in and saves the day, or founded the entire company, or wrote an open source library all on their own.  I get that it’s a good story, but I’m always skeptical when I hear them, because I’m like did they really or is this just the myth we are telling?”

Where do you see yourself in five or ten years? You’ve mentioned you’ll stay in tech, but what do you think you’ll be up to?

I’m really indecisive right now about what I want to do next.  Part of me wants to go and travel and work for a while.  In tech we are really lucky that you can find a job that will let you do this and work from wherever you want, and I think that would be an amazing experience.

Long term, I would like to work on more teams where industry and academia are working together.  I think I’m particularly strong in that area, I learn very quickly and understand how much tech we need to solve a business problem.  I hope to keep doing hardcore distributed systems.  I find this area really fascinating, and I don’t think it is going to run out of problems to solve any time soon.  

Honestly the answer to this question changes constantly.  I guess I just hope I’m always learning and expanding my knowledge. I want to keep sharing what I know, having fun doing it, and feel like I’m working on worthwhile projects that make some positive impact on society whether it’s entertaining people or helping facilitate communication.  

Lastly, what advice would you have for women in tech who are hoping to follow in the same kind of path?

You just have to be stubborn right now.  You have to realize that it’s not really fair, and it’s going to be hard.  I know you aren’t supposed to say that but it’s the reality of the situation.  If you consistently show up, do good work, and find a supportive group of people, you can do well.  It may take switching jobs a couple of times, and that’s ok, you don’t owe anyone anything, find a place that’s right for you.

There’s this great interview with Mindy Kaling where she talks about confidence.  She says “Confidence is just entitlement,” and that you earn it by you showing up, working hard, and earning your entitlement.  

Taylor Swift also summed this up well in her acceptance speech at the 2016 Grammys saying, “but if you just focus on the work and you don’t let those people sidetrack you, someday when you get where you’re going, you’ll look around and you’ll know it was you and the people who love you who put you there and that will be the greatest feeling in the world.”

So I think that’s the advice, you just have to show up, do really good work, find the people who love and support you, recognize it is probably not fair, and be a little stubborn.  

“You just have to be stubborn right now.  You have to realize that it’s not really fair, and it’s going to be hard.  I know you aren’t supposed to say that but it’s the reality of the situation.  If you consistently show up, do good work, and find a supportive group of people, you can do well.  It may take switching jobs a couple of times, and that’s ok, you don’t owe anyone anything, find a place that’s right for you.”