On today’s episode of Build, Maggie sits down with FYI co-founder Marie Prokopets to talk through how to run seriously effective project post-mortems. Marie explains what they are, why you should be doing them, what you can hope to learn, and how to write the perfect post-mortem report. You’ll leave this episode (and your first post-mortem) with tangible takeaways you can use to improve your processes. And hey, they’re not just for product people. Post-mortems apply to everyone and everything. So be sure to tune in for more from Maggie and Marie.
Want Marie’s template for running post-mortems? This Google doc walks you through the steps.
You can get Build on Apple Podcasts, SoundCloud, Spotify, Stitcher or wherever you get your podcasts. Or listen to the full audio version below?
Like this episode? Be sure to leave a ⭐️⭐️⭐️⭐️⭐️⭐️ review and share the pod with your friends! You can connect with Maggie and Marie on Twitter @maggiecrowley @MarieProkopets @HYPERGROWTH_Pod.
Subscribe & Tune In
Full Transcript
Maggie Crowley: Welcome to Build, today I’m super excited because I have Marie Prokopets here with me. She’s the co-founder of FYI, which allows you to search and organize all of your documents in one place. And we actually met a few years ago when I responded to a user feedback survey for her new company. So, I’m really excited to chat today about building products. Welcome, Marie.
Marie Prokopets: Yeah. Thanks so much for having me on. I’m really excited about the topic.
Maggie: Yeah. So, I think we can just dig into it. We want to talk today about post-mortems because that’s something that when we were talking about before, it’s something that I am terrible at. And I often don’t do. So, I was curious to hear from you about sort of A, how you came to a point in your career where this became something that you were thinking about, and then we’ll sort of dig into post-mortems specifically.
Marie: Yeah. Sounds great.
Maggie: So, if you could give me a quick 30 second high level intro to post-mortems and how you sort of came to think about them.
Marie: Yeah. Absolutely. So, post-mortems are essentially a reflection on a project that you completed, or really anything that happened that was of note in your business. And then you basically come up with tangible things that you can improve the next time you do that same thing or something that’s similar to it. They’re also sometimes called retrospectives. And it’s funny, the way that I got into post-mortems was that my co-founder Hiten Shah, he and I used to do some investing. And there was this company we kept meeting with and we were considering investing in. And we had this weird feeling, you know that gut feeling where you don’t know if you want to do it or not and you can’t figure out what it is, but then you also really feel pulled to this.
And one day, I just was sick of this feeling that I couldn’t identity. So, I just wrote this massive data dump, essentially everything that was in my head. All about the situation, why we wanted to invest in them, and then the reality of the product. And kind of where we went wrong. And I’d actually never seen or done a post-mortem, it just kind of spontaneously came out of me. I think all that stuff was built up in my head. And that ended up being the start of our post-mortem format. We actually use that same format today. It’s really about getting all the good and the bad of a project out that’s already in your head. And then improving based on that.
Maggie: Okay. So, I’m just curious. This was something that you were using before you were doing product specifically or in an investment capacity, so not just a product specific thing?
Marie: Yeah. So, you can actually do post-mortems no matter what part of a company you work in, no matter what you do. You can honestly even do it on your personal life. I don’t know if go on a date, it goes really horribly, you could do a post-mortem on that, or an ex-boyfriend, or a car you bought. I mean, you can honestly do them on anything.
Maggie: That’s awesome. Yeah. So, then can you take us through what are the steps or the pieces that you have in your post-mortem template?
Marie: Yeah. Absolutely. So, I’ve got, yeah, this template that I love to use. That I actually used something to, Maggie, thanks to this podcast and looking into. So, there’s something else.
So, first I just start with the purpose. It’s the purpose of the post-mortem itself. So, what does it that you want to discover by doing this post-mortem. Then I move on to the situation. And that’s all the details of exactly what happened. So, if it’s a feature that you’re building for example, you would talk about the feature. You might add in screenshots. You might talk about who was involved in building the feature. You would talk about the research you did. Anything else that’s relevant to what happened in the situation. Then you talk about the results, and just like the situation, the results should be totally unbiased. This is not emotion yet, it’s just here’s exactly what happened.
And the results would be how many users ended up using it, or if you launched it on Product Hunt, how well that went. What customers are saying about it, you could put customer quotes. Any of the analytics that you have. And then you would move on to what went wrong. And that’s really where you get into all the nitty gritty of what didn’t go so well and unfiltered this is where we sucked, or yeah, this is where we didn’t do enough.
And to what we can do better next time. And that’s the area where you can really start to focus on, okay, if you were doing this project over again, or if you were doing something similar what would you change and do. And then the part that I added, thanks to this podcast, is just about action items. So, what are the specific action items that come out of the post-mortem, and then who owns those and when are they due.
Maggie: Mm-hmm (affirmative). That’s awesome. Yeah. It’s interesting. So, I committed before we started recording to doing a post-mortem and of course I did it about 15 minutes before we started this. So, I have one fresh in my head, and it’s interesting that you bring up action items because I think that’s exactly what we sort of did at the end almost naturally, which was we went through all the five steps. We talked about our purpose, the situation, results, what went wrong, what we could better. And then we kind of took down a couple of notes that were all right, now what. What should we do next, what’s our next set of things.
And I think what was really interesting for us was that so, we did a post-mortem on a kick off of a product that we are currently building. So, we had had this meeting last week, and it had been something that we, this team, I don’t think our last kick off, the one we did prior to this one, had been super great. So, we actually had had a really good one, and we wanted to dig into kind of what made it better.
And what was interesting was at the end of this post-mortem every single member of the team slacked me and said, “That was really great, I want to do it again.” And then they said, “I’d like to actually do it for each other step in the process that we use as a team.” Because we had clarified our understanding of those different moments had been different between all of us, and we didn’t realize until we sat down and talked about it.
Marie: That’s really awesome. I’m happy to hear it went so well. And that’s super important that you don’t just do it in isolation, that you actually do it with your team. If you’re not remote, you’d sit down in person and you have a meeting and you talk about it. Because like you said, every single person might have a different perspective or different insights about what happened, what could go better, and also everybody feels included at that point. Nobody’s singled out for doing something bad. And I think the other real benefit of these two, I don’t know if you did it just product people, but because we’re so cross functional, it’s really important that we’re almost setting a good example for everybody else, and doing these types of things. Because then I think engineering is more likely to do post-mortems, and marketing might be more likely to do them because we’re basically this hub.
Maggie: Right. Yeah. So, we did it in the product team, which is me as the acting PM, we had the designer, the tech lead, and the engineer all in the room.
Marie: That’s great.
Maggie: Yeah. It was really interesting. I think just to be really specific, one of the parts of the kick off we have, it’s a meeting where we kind of align on exactly what it is we’re going to build and set some deadlines. And we uncovered that we all had a slightly different understanding of what it meant to scope a project. I had my assumption or my understanding is we scope the problem that we’re solving and then there was another thought about well, we scope when we’re going to do each thing and then someone else would scope how many features we’re going to have. So, we have this really interesting discussion about scope, and I don’t think we’d ever really had that if obviously we hadn’t done the post-mortem.
Marie: Right. And that’s really interesting. It’s like it almost uncovers alignment issues that you don’t even know exist. So, if you had done this post-mortem in a written way and you each kind of added to it, you might not have uncovered all this richness that you actually just … nobody had the same idea of what you were actually building and doing. That’s really great. I’m happy that you benefited from it. Because I know post-mortems are a bit painful to do in a way. We shy away from the mistakes that we’ve made because there’s a stigma, right? It’s almost like you seem weak if you make these mistakes, or maybe you’re [crosstalk 00:08:08] PM.
If you admit that you didn’t do something flawlessly, but in reality if you get to the truth, you’re only going to be able to better the next time. So, I think post-mortems are so important. And they do sound so boring.
Post-mortems. I mean. I saw your tweet today and I was like, “Will anyone be interested in …”
Maggie: I know. Well, even when I set up the calendar invite for the post-mortem, I said, “Post-mortem on X project kick off,” and then in parenthesis I put, “Lighthearted.” Because I didn’t want anyone to think that anything had gone wrong. Because immediately I think, when we think about post-mortems it’s often oh, that thing went so badly that we have to get into a room, talk about what we did wrong, and figure out how to do it better. And like you mentioned, it’s so useful to do it for something that actually went well. And I think the engineer on the team said something. He said, “You know, I think this meeting really pushed us past good enough. We didn’t need to do the meeting, but now that we did it, I think we’ve unlocked an even better way to work together.”
Marie: That’s amazing. And you know, it’s funny, FYI’s a fully remote team. And so, we can’t do that. I actually tend to do post-mortems just in document form, and then we’ll share them around and talk through them at the main product meeting. But I haven’t been doing meetings like you just did cross functional or just the product people where we just go over a post-mortem and actually write it together. And you’ve actually inspired me to start doing that. Because you’re right, it feels daunting to people, they don’t really realize the value. They might put in some stuff, almost like I’m going to just check off the list that I did this. But you’re right. There’s so much value to post-mortems.
Maggie: Yeah. And so, you also sent over an example post-mortem that you had done for a feature at FYI. And I was reading through this post-mortem. So, can you give the listeners a little overview of what this product is? And then I have a couple things I highlighted that I wanted to talk about.
Marie: Yeah, absolutely. So, FYI lets you find your documents in three clicks or less across all the tools that you use. And one of the features that we were building early on was a tags feature. And Maggie, you and I talked about this a little white ago. It’s kind of the expectation, same with email and a lot of other tools. But they just do it. Whether or not it’s actually good for you. So, we were super excited about building tags into FYI. And we thought we had this brilliant idea and we were kind of taken away with our own fantasy about it. And we thought we did everything right until we actually launched it and the results were just abysmal. And that’s when we went back and did the post-mortem to figure out what was it that actually went wrong, and how can we do it better next time. Because that post-mortem, we ended up basically having a bunch of principles that we put into the product development process that were completely absent before that.
Maggie: Yeah. Again, I think was more an example that was more along the lines of what I assumed post-mortems to be, something that didn’t go well and then you had that foreseen function of going back and reflecting on it. And in reading through this document, I thought it was really interesting because in the second you started with the situation just like you said. And then right at the top, there’s this little quote that says, “We got really excited about adding tags because we thought it would help customers, blah, blah, blah, blah.” And I think it’s really interesting, you can already see two sentences in exactly what the problem is.
Marie: Exactly. There’s so much bias in that, right?
It’s not actually tested. It’s like, “Yeah, we thought this, and we thought that, and we thought that.”
Maggie: Well, and I think that’s something that we had talked about which is this is a tool that’s so important because as a PM especially in products or really anyone in that cross functional team, I think it’s so easy to lose your sense of honesty a little bit and assume that you know what customers and assume that you have a full understanding of the problem. And then just sort of jump full force into building something or working on something without checking yourself. And you know, I think I would imagine that everyone listening to this has products that they have worked on or are working on where they’re saying, “Oh, yeah. I know this is going to be great. We think this is a really good idea.” And they’re not saying, “The customers have asked for this.”
Marie: Right. Exactly. I think as a PM, there’s so much momentum to build something, you’re obviously excited about it. You believe that you’ve done enough customer research in many cases and in reality we have to kind of stop and reflect, both as we’re doing the project and after, in order to make sure that we’re not just tricking ourselves into building the completely wrong thing.
I think on tags, so what was really interesting for us was we had, what we realized was we had kind of a fragmented product development process. So, one of the things that we realized was we were absolutely missing our number one requirement for engineering. If we can’t have X, then we don’t want to build this feature. So, you have to make X work. And for us, in tags, we actually had that. I can’t share what it was because really good and we might do it again. But essentially there was this one requirement that we had that we knew if we had that requirement built in that tags would work.
And when we shared it with engineering, at first they were like, “Yeah, yeah, okay.” But we didn’t share just, “This is our requirement.” We just shared that amongst many other things related to the feature. Then later on, we found out they can’t build that part of the feature. And we still … we didn’t stop it. We just kept going. So, that was part of the fragmentation, and now we’ve modified our process to say, “If there’s one requirement or two requirements of a feature, we have to put those up front, or else we’re just not building the feature.”
Maggie: Yeah. Is that also an indication of room to bring your technical team and your engineering team in earlier in the process when you’re doing the part where you uncover that feature?
Marie: Yeah. Absolutely. So, we actually, we have a whole process where we bring in engineering. We have what’s called a feature discussion document where really early on before we’ve done designs even typically, we start to lay out what the feature’s going to look like, and if we have the early designs or any early customer research, we put that in, we talk to them. But we weren’t really getting to the deep part. We would list out 50 requirements that we had, and we didn’t say, “Here’s that one or two must have things that we need.” So, it wasn’t about a lack of communication, it was just about a lack of hierarchy and priortization for them.
Maggie: Right. Yeah. That’s interesting. The way our process works is that we bring in, before we have any designs, we bring in design and engineering to talk about the problem and develop open questions together as a team and that makes sure that if there are any technical considerations or hurdles that we don’t know about we know before we’ve even put any pencils to paper.
Marie: Yeah. I think that’s a great practice. Our engineering team tends to not like meetings as much as I do, probably. I let them not have meetings and do more, since we’re remote it’s easier. Do more via documents and via Slack. And I think this whole conversation’s shown me that we could probably do a better job of just having those meetings. And if I were to do a post-mortem on the whole thing, it’s like we actually just need to force ourselves to have those in person or virtual discussions because if we don’t, we’re missing some of that alignment.
Maggie: Yeah. That’s interesting. We are a fully I guess co-located team. So, we default to having face to face conversations. And it was interesting, even in the post-mortem that I just did, in the kick off one of our team mates was out of town, so he had dialed in from remote. And that was definitely in our what went wrong column because he felt like he didn’t have as much clarity as he would’ve had had he been in the room.
Marie: Oh. That’s really interesting. I mean, yeah. That’s the other thing that when you do a post-mortem, you realize the situation is so specific to your company and the team that’s working on it. It’s vastly different if you’re co-located, remote, if one person’s remote.
Yeah. That’s really cool that you were able to uncover all that. I was worried. I saw your tweet, it was maybe an hour before. And I was like, “Is she going to have enough time to do a post-mortem.”
Maggie: The team was very nice to me in that they agreed to do it without any preparation and at a high speed fashion. Basically what we did, it was … we just I wrote down purpose, situation, results, what went wrong, what we can do better on the white board in a bunch of columns and then just high speed wrote down and took a picture of it.
Wow. That’s great. It’s nice, you don’t have to do it like I do them formal documents. But you don’t absolutely have to do that. You can do the white board. You can do a recorded discussion on Zoom, you can do something in Slack.
Maggie: Oh, yeah. That’s a good point. I wanted to bring up something else that I read when I was looking at the post-mortem that you sent over. And I think it’s interesting just in general as a product person, there’s a, in the what went wrong section, there’s a little bullet that says, “We were jumping ahead to secondary problems before we had nailed helping people find their documents,” which is sort of the first problem that your company solves. And I just think that’s such an important thing to call out. Because I think we probably all do this in some regard. Where I’ll forget how hard it is as a customer to use a tool and how just the basics of a product are so important, and we always want to build the next cool thing. So, I just thought it was so fascinating that that was something that you had called out specifically.
Marie: Yeah. I think you’re right. I think you can get lured into that anytime. Whether it’s your first month after launching or you’ve been around after years. You can lured into the desire to keep building on more features that you don’t need. That’s how a lot of us end up with these bloated roadmaps. We actually have a roadmap that it’s aspiration, right? There’s all these things, but in reality we can’t really move onto those features until for us we nail making a product that teams love and we nail helping people find their documents quickly. So, for us, it’s having them do tags for example which requires a bunch of work and it’s not actually helping them find their documents without them doing work. That’s not something we should worry about yet until we’ve solved that primary problem.
Maggie: Right. And were those some of the principles that you had mentioned earlier that you guys aligned around after having gone through this discussion?
Marie: Yeah. So, we actually have three product tenets that created, and this is what I meant by our product process was fragmented and the post-mortem helped us realize that. We had these three tenets, they’re really researched backed. It came out of a 300 page deck that I did on strategy for the company and the competitive space. And what we ended up doing is we had them and we forgot about them. We never weighed every feature against those. And so, as soon as we did tags, I looked at the three tenets, and I realized that I think two of the three, or even three of the three were not actually fulfilled with tags. And so, for us what this helped us do is really realize for our product development process, what are the must have things that we need to do for every single feature that we ship. That if we forget about we’re probably going to ship the wrong thing.
Yeah. And the other big thing that we realized here is that we focused on user testing when we built tags. And user testing is amazing, I don’t know, I’m guessing you do a bunch of it too.
Yeah. So, user testing is great. We love getting recordings of customers talking through the product, whether it’s the actual product or designs. And because we just focused on that, we got complete false positives because people thought tags were great, the designs they looked really good, in theory you would want to do this work. But in an actual customer environment, people aren’t going to do that work yet. And they might say they want the feature, but they don’t actually. And what we had done is we user tested tags in onboarding we did five different user tests where it was in onboarding. What I mean is in the home page.
And people got dropped onto it when they got into the interface. And we did a bunch of user tests of it in the interface, and it just looked like this was going to be the best feature we were ever going to build. And it completely wasn’t. And the other problem was we never did any early access for this feature. So, the first time that customers used it was when we shipped it to production. There was no let’s let our actual customers use this and give us feedback before we ship.
Maggie: That’s so important to remember and that’s one of those things that I can totally see how you would skip it, especially if you had such strong results from the user tests. We did something where we honestly just grew out of me not having enough time to properly run a beta, I just asked a couple of customers to use a new thing and film themselves and send me a video. And I got these amazing results because I hadn’t been able to give them any background, I didn’t intro it, I just said, “Go to this thing, this page, and record yourself building something and let me know how it goes.” So, it was a really interesting way to get that feedback, as unbiased as I could get at the time.
Marie: Yeah. Absolutely. You can do a post-mortem on that, by the way.
Maggie: Yeah. Maybe I should.
Marie: Yeah. They don’t have to be super long or anything. I think you’re right. The post-mortem can even uncover things like you only talked to customers in early access that already love your product and that are really kind of evangelizing it already. And they might be more excited than the average user that just started, right, that it’s their first month or they’re not quite as happy with you. So, I think the good thing about doing these is you can really get to the nitty gritty details and understand exactly what went wrong.
Maggie: Yeah. Yeah. I also think about understanding what led you to make the decisions that went right as well. Because I always think about okay, we’re scaling really fast at Drift and how do we share what we’re doing with the other teams so that everyone can be scaling as fast. So, every time something goes right, I want to think about okay, this went right for us, but what can I learn from this that I can share with the rest of my teammates to help them also do it right. Just because I think it’s so easy when something goes right like the engineer said, “That was good enough,” and we would’ve kept running if we hadn’t had a reason to stop and force ourselves to think about it.
Marie: Yeah. I think that’s a good point. We tend to do post-mortems on things that went wrong, and in part that’s because of the name, post-mortem, right. It’s an autopsy, right, on something that died. But in reality, if you do them on a project that went well, then you can actually rally the team around all the amazing things that happened and use it as a way to kind of even give everybody shout outs.
So, I think you’re right. We’re really quick to jump to these negative post-mortems, but if we create a process where you do them for anything, then you’ll actually end up with a bunch of positive ones too.
Maggie: Yeah. We definitely left that room feeling really good as a team, feeling really positive. And considering it’s something that we’re actively working on, I think it was probably a nice thing to do mid-build because everyone went back to their desks sort of hyped up on the thing that we were doing.
Marie: You’re inspiring me too. I think after this, I’m going to have a bunch of action items.
Maggie: Yeah. Go do some post-mortems.
Marie: Totally, on positive things.
Maggie: Definitely. Okay. So, I think just for the listeners, right, it’s so simple. You just have to have your purpose, your situation, results, what went wrong, and then what you can do better next time. And then you added the bonus of action items.
Marie: Yep. Yeah, exactly. That’s it. It’s really simple. And you can create whatever kind of format you want. So, at the top of my template, I just recently added some kind of cheerleading type stuff and some go wild, be nice, be honest, don’t sugarcoat, it’s okay to make mistakes, it’s even better to learn from them. So, just whatever it is that helps you feel free to write the post-mortem, that’s what you should put in the template.
Maggie: Awesome. And I actually read those comments out loud to the team because I thought they were part of the official template. So, that helped.
Marie: Oh. I’m happy to hear that. Yeah. I got some feedback from one of our PMs that they really liked it and it helped them feel empowered to do the post-mortem.
Maggie: Yeah. Definitely. Okay. So, we’re almost out of time, but I have two more questions for you. I want to know what are you reading or listening to that you’re recommending to people that we can all learn from?
Marie: Yeah. Absolutely. Right now, I’m actually really into spiritual books less than business books.
So, right now I’m reading Autobiography of a Yogi by Yogananda. It’s my second time reading it. The first time I wasn’t able to finish it for some reason. So, I’m actually finishing it this time. And one book I always recommend that I’m absolutely obsessed with is called The Artist’s Way by Julia Cameron and it’s all about creativity. It does have a spiritual bend. So, if you’re not into that, I would avoid it. But it’s all about unlocking your creativity at work, outside of work. It’s one of my favorite books. And then I do have a business book coming up next that I’m going to read, my co-founder is raving about it. It’s 15 Commitments of Conscious Leadership. So, I’m excited to get into that one. It seems both spiritual and business.
Maggie: This is good because I usually get the same set of five books everyone recommends. And all of which I just haven’t read. So, now I can add new books that I might actually will read this time. And then what’s one or two pieces of advice that you would give to people in products? Obviously, you’ve had a really wide ranging and successful career. So, what advice would you give to people listening who want to found a company, want to get into products, any of that?
Marie: Yeah. I think the advice I would always give is just do more customer research. It can never hurt you. And if anything, you’re always going to discover more about what the customers’ needs are, what they want, what they love about your product. And because of the tags post-mortem, the other piece of advice I always give is don’t bias yourself by doing one type of research. So, you should always be doing a variety of customer research, whether it’s interviews, and user tests, and competitor research, and surveys, and looking at your analytics, but don’t just bias yourself with one.
Maggie: Awesome. Well, Marie, thank you so much for coming on the podcast. I really appreciate it. This was super helpful and awesome.
Marie: Yeah. Thank you so much, Maggie. And on my end, I learned a bunch too.
Maggie: Yeah, we should just do a post-mortem on this podcast.
Marie: You’re right.
Maggie: Awesome. Well, everyone listening please leave a review, give Marie six stars. We need some more shout outs in the comments, and thanks very much.
