Product Coalition Product Management Podcast

EU Tour #6 Learning from your mistakes with James Harding

March 20, 2020 Jay Stansell, James Harding Season 4 Episode 6
Product Coalition Product Management Podcast
EU Tour #6 Learning from your mistakes with James Harding
Chapters
00:00:00
Introduction
00:03:07
Cardiff quiz
00:06:42
Background
00:07:29
Connection of engineering and literature
00:08:35
Mistakes and learnings
00:12:57
Relationship between Data and research findings
00:13:43
How the multilingual side affects the product design from the engineering perspective
00:15:36
User's VS Costumer's surveys
00:16:45
Do you need to educate your costumers?
00:17:34
What can you not ask people?
00:18:17
What are the learnings?
00:20:17
How to work with the costumers to get the why's?
00:22:03
The top tips to build a top survey
Product Coalition Product Management Podcast
EU Tour #6 Learning from your mistakes with James Harding
Mar 20, 2020 Season 4 Episode 6
Jay Stansell, James Harding

Listen in as Jay Stansell and James Harding chat about learning from your mistakes.

To support the bushfire affected wildlife and communities of Australia that are mentioned in this episode head to bushfire.productcoalition.com

To get pre-release access to all Product Coalition podcasts, product management mentorship, product management interview practice, and product management resume reviews, visit platform.productcoalition.com

Show Notes Transcript Chapter Markers

Listen in as Jay Stansell and James Harding chat about learning from your mistakes.

To support the bushfire affected wildlife and communities of Australia that are mentioned in this episode head to bushfire.productcoalition.com

To get pre-release access to all Product Coalition podcasts, product management mentorship, product management interview practice, and product management resume reviews, visit platform.productcoalition.com

Support the show (https://platform.productcoalition.com)

spk_0:   0:15
Welcome to another episode ofthe product Coalition European tour. I mean the bonus city of Cardiff, where I'm running the Cardiff Siris on down. Very excited to be joined by James Harding from Du Po. Welcome, James. Great to have you here. If you're watching on YouTube, you'll see that we're in a bit of a strange location. We're actually in an old vault, which is a great recording location, and hopefully from sound perspective also makes for for great listening. Now, before we get started on first, I want to give a big thanks Tio do poll for the introductions to all the guests in Cardiff. Do pose a global online survey provided that has recently switched to a pay for value model, meaning you can get started with their real time, multi lingual and embed herbal surveys for free today. Just go to do pole dot co dot co. That's t double o p o w l don't see now this tour and every single podcast episode that I'm recording is dedicated to raising awareness and support for the Bush fire affected communities, wildlife and volunteer firefighters back in Australia. If you enjoy this episode, please show your support for these causes by heading over to Bush. Fired up product coalition dot com. Now I'm travelling across five cities to interview over 50 product leaders like James to gain insights, knowledge experience to share with you the product. Coalition Global Community If you just discovered the product. Coalition Welcome. We are a global community of over half 1,000,000 readers, 6000 slap members and thousands of podcast listeners. Now, before we get stuck in, I need to give a huge thanks to a couple of brands and people that have made significant donations to support the cause that Bush fired up. Product coalition dot com First up is User Pilot, which is a code for a user on Borden Adoption tool designed especially for product management teams. Use a pilot helps to increase conversion user retention rates and reduce churn by guiding new users to their first Har moment with interactive walk throughs. Contextual product tours on on boarding checklists allows product managers to build four customizable behaviour. Terrific triggered enough experiences with a simple visual editor going to use a pilot dot com to book your demo and get a free trial show. Big Chuck is the intentional product manager Schaub. It's a Google product manager, and he helps product managers become product leaders. They can have careers that they're proud off head to intentional product manager dot com and sign up for show because free class on the habits that turn product managers into exceptional product leaders and help them move through their careers. Fast product led teams like Mixed Pan on Flex Paul know that the best time to capture engagement is when it uses already inside the product. That's why they use chameleons. Dr. Feature Adoption, build on boarding flows and gather user feedback. You can give it a go at tri chameleon dot com forward slash success to individuals I'd like to thank for that for their donations. Is Rich Marinoff and Chris Miles James. We could get stuck in, and I'm looking forward to this session. We're gonna be talking about learning from your mistakes. I hoping you got a few to it to rattle off. I know. I certainly have. So this is gonna be good fun now you would have seen in the Melbourne podcast bit of an icebreaker around a bit and Locals Guide. And in the Sydney Podcast we had had the pop quiz. So for the European tour, I've gone a bit different. And for Cardiff icebreaker I've gone with Is it Welsh or No? Okay. All right. So first up, we've got a couple of products and gonna get your read on whether you think they're Welsh or not. Okay. All right. So first up is the radar. I'd say that's Welsh. Back up blind. Guess I guess you got a 50% chance and you've now added so Swansea born Edward Taffy Bone was one of the one of the pioneers in the discovery of the radar. He built the first radar transmitter in the 19 thirties, and this inspired him to build smaller transmitters which could be fitted into aeroplane cockpits without radar. The Second World War could have bean a very different storey. So next up is the memory scanner. It's

spk_1:   4:27
a not Welsh. Any reason again show. But I just don't feel like Felix bigger than well,

spk_0:   4:34
so in the nicest way. Ah, where do you think that us? Us? Yeah. I could see right, but you'll be right. It's not Welsh, okay? It's our friends a little bit further north in Scotland. Oh, wow. The breakthrough for the memory scanner was made by a team working at the University of Aberdeen. Ian 1980. The team obtained the first clinically useful image of a patient's internal tissues. There we go right next up of Got some words and even their Welsh words or saying or I just made him up this morning. How is your wealth Roche vocabulary. I lived there for five years, so I know a few little things, right? Okay. All right, let's go. Because I'm hoping fully integrate yourself into Welsh society in language. Let's let's give it a go. All right. Okay. First up, we've got in. Coleman call sounds made up. Coleman core like called Moon Cole. Over. There s o. It means him over there. Really? For the most people listening and probably got the accident completely wrong with a bit of company thrown in throwing you off. It's the company's cut me Ozzy twang that's thrown it out. All right, let's get the next one actual e gorgon. Get your chiu l. E. Grogan.

spk_1:   6:07
I think I once made up

spk_0:   6:08
about it's Welsh. Actually, Gogan is a big round thing with horses on otherwise known as the merry go round, so you could slip that into conversation next time. Next time they start, he's down the having card. All right, that's that's the icebreaker done. Well done. You got one out of four. Pretty cool that e. I think you have another five years in where I was cool. Let's let's get stuck into learning from our mistakes. Jones. Before we do that, would you mind sharing for the audience a bit? Bit of your history, Career background set some context for us. Sure. So right

spk_1:   6:54
now I'm the lead developer at DuPont and I didn't used to be in design and development, though. I originally did my master's in literature on DH, then self taught myself whilst I was travelling around Asia, building websites on the side and doing some freelance thing there. I joined do poll around just over two years ago now and sort of hit the ground running, had to learn a lot but and loving it, really enjoying the product scene.

spk_0:   7:19
Also awesome. Thank you so much. It's It's always interesting to hear people's stark and where they are today and how that path change. Was there any Connexion at all to engineering from literature back in the literature, that literature that has a lot of

spk_1:   7:36
Connexion in terms like the research you have to do and learning to learning to be a researcher, right, Googling things and trying to find, you know, the answer to problems. A lot of what we do on a day to day basis. And, you know, writing good copy is also a really important part of framing your happened of helping your losers understand what you're doing and what they need to do. So having a good command of language from that can can really help. Originally, I wanted to be a travel writer, which is and why, you know, I built my own block, and that's how I learned how to build things and build websites. That's kind of really where it started, so it doesn't feel like a normal sort of transition. But there was a lot of sort of like goal posts on the way there.

spk_0:   8:16
I imagine you're you're commenting in the code. He's beautiful. I like to think so. Well formed verses, little summit, something Coach just just gives it a bit of spark for the engineers reading for Fantastic. So today we're gonna talk about learning from mistakes. Get us started. What mistakes have you bean making and learn from when it comes to product decisions. So far, one on ones that

spk_1:   8:49
comes to the forefront of my mind is a recent feature that we've released, which is open text. And there's a new question type for us on DH. We've been holding off on building it for around three years now. We've had a lot of requests for our users. It's something they've released of push towards, and we've been a bit opinionated in holding off from and and the reason why we've now have changed our mind and realised that maybe was mistakes hold off and there's a lot of things involved in that. And I think the the main thing is the fact that we needed to understand the solution that we were trying to solve for I use is we've got two sets of people who use our platform. We got the respondents and we got the people who create the surveys and we felt the response rate was a really important part of you know what our users wanted from the platform Getting more data is is better. Um, but you know Ah, get. And one of the things that we we learn as time goes by is that a lot of what I use is one is actually being out to dig deeper into the data. Robin just having more of it. Raka. And so, while we were protecting the platform, open taxes should say has a lower response rate then, of all of the sort of question types we would select from an option because there's more of a barrier

spk_0:   10:07
in place. So these are the big input boxes. That's when the forms, as opposed to choosing from Breda bottom buttons or Okay.

spk_1:   10:16
So, um, by switching and adding the open tax, we knew that response rates would drop down, which would be worse for our users because they'd let get less data. Right? That's it. After talking to our uses a lot more of learning from the last three years we've released come to see that even though they might have less data being able to understand one or two's off really cool comments from from users could be really impactful for them, right? Show these around you know it's the marketing team shown around board meetings, that something, and it could get a lot done. Alot of action could be made from

spk_0:   10:47
them.

spk_1:   10:48
And so that was this sort of a mistake we made in understanding where the rial core problem lied for them and what they what they needed. Response rates still really important. But it's just not always the main thing. You've

spk_0:   11:01
gone from being purely quantitative data driven to now, mixing up with a bit more insight along the way. Absolutely, and you're finding it's a worthy trade off is what I'm hearing for a drop off, right? But if you get some quality around it, fantastic and, uh, can you tell me on the same line of thought, what else could you looking to do more off or less off? So I think

spk_1:   11:32
using metrics to really understand and make good product decisions is something that we want to do a lot more of very reason. It's only a small detail, but we've built charts that can be downloaded for the data that people have on the platform on DH, one of the great feature really great adoption rate. But one of the things that we had a problem with. This is the labels on the chance right? Because users can have longer label is they want, causes problems with design. So we made a decision to snip them off a certain amount of characters. That was kind of an arbitrary figure of 22 and we've had a lot of requests for our. How could make this better? Thiss isn't good enough for us. Some of our users will have a Welshman English translation side by side, and it cuts off one of the translations completely. So one of things I did this week was to really get to grips with how much the average character rate of our users are and dig down to how we can build this to be the best for the most amount of them. So a 5% So doing that data sort of analysis on pulling the inside. We've now made it 35 characters, which will account for almost all of thie. Apart from this sort of like top end. Using those sort of things will really help us been going forward, and it's saying that I haven't done is much of in the past when making those decisions for kids and she hasn't come out.

spk_0:   12:59
Can I ask about your lead engineering role there on DH? You're jumping into the data, but you've obviously got some product direction. So how does this relationship between the day of digging and insights that you're finding that you just talked about and product come come together and working with the product team and U ex? Obviously, by the sounds of it as well, Well, we've got quite small team. So a lot of those different

spk_1:   13:23
roles this done by me and the three of us, really, And so, in terms of the relationship between them, I think it's just a case of trying to balance them. You know, the different factors in there on getting the time to do that. But the due diligence and to get that data and really sort of understand what it means.

spk_0:   13:43
What, like you mentioned earlier, was about the multi lingual side of him as well. Could you tell us a little bit how that maybe effects the product design, and particularly from an engineering perspective, how do you handle that? It's really hard because you don't

spk_1:   14:00
over an engineer the platform right? Don't give too many options to the years I've been creating a the hard thing about multi linguist trying to maintain a really good user experience. It's also got a lot more overhead on the development side of things, even from like doubling up on the translation strings. It's a lot of extra work to do on Earth and slow down release. We want to work in like a really lean of methodology on DH when we're waiting, you know, an extra week to get translations. Before we could get a feature out, it can. It can hold us back.

spk_0:   14:29
So it's It's been

spk_1:   14:30
a challenge, I think. But it's also one of our strengths of platform. Again, it's about balancing like this is something that I use is really need, and one we've got to be the ones to find a way to do it well for them so that it is. It maintains that simplicity and easy to use features without putting the bed on the

spk_0:   14:52
Yeah, yeah, on DH, the product Is it designed for being embedded in other sites? Yes,

spk_1:   14:58
aren't saying way. Recently, Belt could be embedded into Blog's and tio different websites on DH. One thing we've had a lot of success with recent isn't articles of, like online media and engagement with it is has been really high on. One thing it's been really good at is getting emails, right? Okay. We feel like we found that by asking a few questions of people again. Not the open text, because this concert at a barrier there, but easy engage from questions and people feel like they've invested in something and more likely to sort of write that e mail at the end. Right now, something I use is some Really appreciate.

spk_0:   15:33
Right? Okay. Okay. This is really, really good. And so working and designing the product for users that complete surveys vs customers that I suppose they're paying you for the service. Do you see different needs from from both? Is there any conflicts that

spk_1:   15:53
absolutely, I think you know, for a respondent, it needs to be as simple as possible. And you don't want to find any sort of barrier in the way of them answering. If you want to get that completion done and get the best answer for them and at the same time, you know, if that data isn't valuable to the user that's collecting it. It hasn't got its not worth that much for them. So really, there's a there's a conflict because we need to balance getting a CZ much data as possible without getting some data that doesn't have much meaning

spk_0:   16:26
to. And I know from my own experiences to have a career of building any sort ofthe surveyor online form, um, having to keep myself in cheque of Why am I asking expression like if I had 1000 responses to this drawer force answer. What would I do with that data? Do you find yourself having to educate your customers? Absolutely. Contents right? OK,

spk_1:   16:50
I think it's really a real difficult thing to get proper factional insights, you said. And that's one of the units of the missions that we want to do is help. I use it to get those actual insights or love. That's education. So we've recently released on your block, and we put a lot of work into into getting that out there. Marco CEO does a little work on that and that sort of educating our users on to why are they asking their surveys? What questions can they ask? And how will that affect their uses? For instance, talking about the open tax and how you know from the data we can see that it drops, you know, 10 15% of your users you might lose. Just from asking that question. Do you want to ask it at that time? Do you want to start with it and trying to help them? It's off Constructor seven The best way possible.

spk_0:   17:33
Yeah, I imagine, for a lot of business to say once, um, a lot of deep data around people on DH. Imagine is quite a bit to think for a river gasto. The timing off went to ask some personal questions on DH when not on. Imagine you've also got regulatory. I've hit of what you can't ask people. Maybe, I think,

spk_1:   17:53
yeah, that's been something that's come up a couple of times recently, and it's It's an interesting one. You know, there's there's a lot to think about when you're dealing with data. Ah, nde, You know, making sure that's protected on. You can see that data from our from outside. You know, we've had questions about like you could see things on the platform and what not say it's something to think about.

spk_0:   18:17
What, what have you learned since since entering into this role that you didn't know before that now you would give us tips or advice for anyone Move in particular, Supposed from an engineering packed out practise that's also wearing a product that, like you are

spk_1:   18:32
absolutely, I definitely tend to over over engineer things on DH. One thing that's really help me is he's asking why Mo? So I've almost annoyingly five times keep asking why? So you can really understand the problem that you're dealing with a solution that trying to solve in. One thing that we had a couple of years ago was a lot of requests for Grouping and Categorising polls that people had been answering. That seems like a fairly simple, easy featured belt and something they'll be useful. We dug into it more. If we asked why many, many times on DH found out that you know what the person was doing was just duplicating the same poll many, many times. And what we really wanted was to do the same poll many, many times. That was what was filling up their account with all these extra things they needed a group in organised. So we eventually built feature called Recurring Pulse, which allows them Tio Sian Track over time or across groups has been a really great feature. People really enjoy it, but if we have taken at face value, you know, basically would have ended up with the right thing. And I think on top of that you can get into the state where you're build. Everything that comes in, gets asked and probably a few more and you're kind of really missing what, what's being asked of you. And so that's something I've really had to learn some time because I get excited about new features and I just want to dive in and build them

spk_0:   19:52
a CZ. We know it's It's the building. Then you got to support it and you gotta manage it and your regression test it. And I love that Storey Teo here that you found particular instance where you've prevented making the mistake as opposed to learning from mistakes, you know, from those mistakes and their year on a preventative path of being able to prevent cracking features that are loved. Can I ask, how do you work with your customers to get in tow, wants those wise is that do you serve your customers? Is it quality of more face to face interviews? What's your approach and do a

spk_1:   20:32
number of things? One is to sever them when we've got the tools that I wouldn't say. It's a useful way to do it, but a lot of our interaction will love. My interaction comes from support requests and then digging deeper into those. So I have a support request. If I can't do this, how do I do there so and something? And when we dig in deeper on using to confidence, we could get a lot of great insights from that. Our customers also really engaged so we can organise all interviews and phone calls so I can really get a chance to chat with them and get feedback that way. And that's been invaluable for us. You know those real conversations because often you know when you've got just data just met drinks doesn't always give you the complete back picture. You kind of gotta have both.

spk_0:   21:18
I can imagine strategically for the business. The with the amount of data that you're collecting the data science

spk_1:   21:25
opportunity is gonna be huge. Yeah, it's something that we've looked into a lot on DH. There is so much opportunity there, you know, we do some work with sentiment analysis in the moment, which is something that you know, I'm really enjoy working with and and trying to understand more. But some of things we were talking about earlier with constructing a good survey, this is something that really interests us from a day to perspective. Can we sort of from that data started? Understand how what a good question looks like? Can we write that as soon as they use is sort of building it and things and, you know, and I think that's what we would like to work towards a lot more in the future of leveraging that,

spk_0:   22:04
Can I ask for the product managers listening? They're always surveying or running quality of workshops, et cetera and asking questions off users and other teams, et cetera, working so close to the surveying questioning space that you are on a day to day basis. What some top tips for building a good survey that product management should should do that. That might be obvious but they still forget about or ignore What would you say?

spk_1:   22:32
And the thing you mentioned? A one good point earlier, which is really understanding what you want to ask what you're trying to get from this, you know, have a hypothesis that you're trying to test. I think that's a good place to start. I think on another level, keep it short. You know, people don't want to be answering hundreds of questions. I've done some surveys in the past where you know, I've got 20 pages in. I just I'm just gonna give up on it. You know, this is an important thing. Ah, again, try and make your date of wherever can Quantitative. You know, it's easier. Teo work with that data and draw insights from on Only dig down once really necessary.

spk_0:   23:10
Awesome. Awesome. What product development approaches air you admire. And who else in this space you watching?

spk_1:   23:17
And I mentioned that we use them earlier intercoms aerial, Real great pro development company that I love. They've just got a great mission making the Internet personal on DH. I think having a good mission to align behind is something that really creates good products. I think. I think we're working more towards a really great alignment phrase that can take you forward and help decide what features you build or don't you know? And it should be something that will cut some things off and make sure that some things are included. I think they've nailed that on into coming. That's what the hell's that is great and useful product.

spk_0:   23:56
Yeah, there's certainly gone from strength to strength in that I think is probably about 45 years now, throwing them on and again, doing a lot. We've obviously natural language processing and automation, etcetera. So yeah, definitely an exciting one to watch. Thanks so much. Thanks a judge. It's been great to talk through, You know, making mistakes is owning up to mistakes and being able to reflect on the mistakes. I think it's very important for anyone in any role, to be honest, but what I particularly tip my hat to is when people are willing to have a conversation about the mistakes and the reason for me wise, they put themselves in a vulnerable space and always have a lot more respect for someone that's willing to put themselves to somewhere vulnerable. and talk about mistakes and be able to reflect either at home with family or appear at work, or just generally with the industry as you're doing today. So thank you very much. It's Bean being great stalk through. It's

spk_1:   24:54
been great. Thanks for having me

spk_0:   24:55
a pleasure. If you've enjoyed this episode or any of the episodes with product Coalition European, too. Please do remember on committing this time to meet with over 50 people to raise awareness and funds for the Bush for affected communities off Australia, as well as the wildlife and the volunteer firefighters. So please consider donating to those causes I wrote bushfire product coalition dot com until the next episode from Cardiff. Okay, you say.

Introduction
Cardiff quiz
Background
Connection of engineering and literature
Mistakes and learnings
Relationship between Data and research findings
How the multilingual side affects the product design from the engineering perspective
User's VS Costumer's surveys
Do you need to educate your costumers?
What can you not ask people?
What are the learnings?
How to work with the costumers to get the why's?
The top tips to build a top survey