Product Coalition Product Management Podcast

EU Tour #3 Building data products inside a large corporate machine with Peter Sueref

March 17, 2020 Jay Stansell Season 4 Episode 3
Product Coalition Product Management Podcast
EU Tour #3 Building data products inside a large corporate machine with Peter Sueref
Show Notes Transcript Chapter Markers

Listen in as Jay Stansell and Peter Sueref chat about building data products inside a large corporate machine.

To support the bushfire affected wildlife and communities of Australia that are mentioned in this episode head to bushfire.productcoalition.com

To get pre-release access to all Product Coalition podcasts, product management mentorship, product management interview practice, and product management resume reviews, visit platform.productcoalition.com

Support the show

Jay Stansell:   0:17
hi, everyone, and welcome to another episode of the product Coalition European Tour podcast. I'm in the bonus City of Cardiff today, where I'm very excited to be joined by Peter Sueref. Peter, welcome, thank you. Looking forward to chatting through today's topic, which is building data products inside a large corporate machine like Centrica.

Peter Sueref:   0:37
Yeah, Centrica. A very big company, 30,000 employees. And somehow we still manage to just about get data products out now and again.

Jay Stansell:   0:44
Good. I'm sure there's plenty of learnings we could all learn from, from from that looking forward to going through that. But first I need Teo. Get first to give some thank you's first up a big thank you to Do Pole dot CO for the introductions to all the guests here in Cardiff Do Powers, a global online survey provided that has recently switched to they pay for value model Minion, you can get started with their real time, multi lingual and embed herbal surveys for free today. Just go to do pole dot co. That's d double o p o w dot com Now, this tour and every single podcast episode is dedicated to raising awareness and support for the Bush for affected communities and wildlife in Australia. If you do enjoy this episode or any of the episodes on the product Coalition European to consider showing your support at bushfire dot product coalition dot com to support if the volunteer firefighters, the wildlife or the community's off Australia as I'm interviewing 50 product leaders across five European cities, and I'm gonna gain insights, knowledge and experience to share with you the product Coalition Global community If you've just joined us, welcome. We're a community of over half a 1,000,000 readers, 6000 slack members and thousands of podcast listeners. Now, before we get stuck into the episode, also need to give thanks to some brands and individuals that have made significant contributions to the to a fundraiser for those Australian communities, wildlife and volunteer firefighters. First up is User Pilot, which is a code for a user on Borden Adoption tour, designed especially for product management teams use a pilot helps to increase conversion user retention rates and reduce churned by guiding new users. So the first harm moment, with interactive walk throughs, contextual product tours and on boarding checklists, allows product managers to put fully customizable behaviour. Trickett enough experiences with a simple visual editor. Go to use a pollock dot com to book your demo and get a free trial show. Bic Cheuk is an intentional product. Minjae Show Bit is a Google product manager, and he helps product managers become product leaders and have careers they could be proud off. Go to www dot intentional product manager dot com. Sign up for Schaub. It's free class on the habits that turns product managers into exceptional product leaders and help them move through their careers. Fast product led teams, like mixed panel inflect support, know that the best time to catch engagement is when a user is already inside the product. That's why they use Chameleon to Dr Feature Adoption, built on boarding flows and gather user feedback. You can give it a go try chameleon dot com forward slash success. I'd also like to think Rich Mironov and Chris Miles as individual donors to the fundraiser as well, Right, Peter, Let's get going before we get into the world of building data products inside corporate machines, specifically large corporate machines. We've got the ice breaker. So in the Melbourne series it was a bit of a local guide to Melbourne with the guests in Sydney. It was a pub quiz. So for Cardiff in and all the different cities on visiting, I've gone a bit more local. So the icebreaker game today is is it Welsh or no? Oh, God. Okay, let's do it. Let's tio you Welsh born and bred.

Peter Sueref:   3:44
I'm well, I'm born in Wales. I've got Greek Parentsage Greek history. So my my Children speak Welsh.

Jay Stansell:   3:50
Wow, right. Okay,

Peter Sueref:   3:51
well, so this could be interesting. I didn't expect to do very well.

Jay Stansell:   3:56
All right. Okay. Well, first, the first up, we've got a couple of products. So the first product this could be controversial could create quite a bit of a storm on social media Dennis Welsh or no.

Peter Sueref:   4:11
So there's something in my in my brain which says, actually, the guy that maybe came up with Guinness Region blows a Welshman. I've kind of got this big lake knowledge. I think so. I'm going to go, actually, yeah, this and Welsh history, they with Guinness.

Jay Stansell:   4:23
Let's have stemmed from, like some Welsh folk law, So training from pub to power across Yeah, the value of my misery s so my friend. The Internet says that you may think it's an Irish institution, but legend has it that the recipe was allegedly bought by Mr Arthur Guinness from a tavern keeper in Holly Head, North Wells whilst evil waiting for a ferry and taken over the sea to Ireland, where it became this iconic symbol is today.

Peter Sueref:   4:56
Well, there we go. That's amazing On DH, the related linked to data science, which I'll talk about, is the guy that invented the T test. This famous statistical test work for Guinness on DH. He was thie kind of in house statistician that used to figure out where the Guinness met its quality control or not. So if you could

Jay Stansell:   5:14
give us an eye injury

Peter Sueref:   5:15
in Dublin is little plaque on the wall there. Everybody ignores, and it says students T test was created here. A little bit of stats, folklore.

Jay Stansell:   5:23
I tell you what, When I do a doubling to get on it, I'm gonna get to that plaque whom you are linked and I'm gonna take that picture of me on that black find Brilliant. See? Look how much value of we already creating this episode. This is brilliant. All right, Then the next one next product. And I'll tell you what. He could be controversial as well. But I'll get going with it in the refrigerator.

Peter Sueref:   5:47
The refrigerator. No, I'm not. I have not heard anything related to Wales there, so no, that's That's a complete mystery to me. I'd be really interested to hear that.

Jay Stansell:   5:56
What do you think

Peter Sueref:   5:56
it's from? I mean, I was going to say Scott links. They invent lots of things restored to really cold. There's the party's not much need for it may be a southern country, right? Yeah. I mean, some of Southern Europe.

Jay Stansell:   6:08
You'd be right with Scotland.

Peter Sueref:   6:09
Okay, well, yeah, they invented a lot of stuff.

Jay Stansell:   6:11
Scotsman William Cullen is the man we have to thank for the invention of the refrigerator. Cullen demonstrated his discovery at Glasgow University in 17. 48. That the time knife it was made to commercialise. You mentioned a reason I say controversial is from my memory. I'm sure we had the refrigerator in the London podcast. Serial rights. I just can't remember if it was attributed. Maybe it was attributed to a British invention. Yeah, hands the happen if you're in Wales or Scotland If you do. If they do well, that's their

Peter Sueref:   6:39
British. But if they do poorly, then Andy Murray, Scotsman Gareth Bale, Welshman of something. So

Jay Stansell:   6:44
yeah, very true. Very shape. All right. Next up, S o. Yeah. Kid's gonna love love this one. We've got some A bit of a wash linguistic quiz going for you. So the quizzes you got to decide whether the what's the word that I'm saying? Whether it's Welsh or whether I just made it up this

Peter Sueref:   7:04
morning I'm going to get thrown out

Jay Stansell:   7:06
whales at the end of this. Why you're doing this, Tio? Okay. 1st 1 is Mana Mana monkey. I don't know why I did it in a Georgia accent, man. A man. A monkey?

Peter Sueref:   7:22
That can be a real word. No, I'm going to go with you, mate. You made that

Jay Stansell:   7:26
man a man. A monkey is my as well in Welch. Might as well, madam. I didn't learn that in well, throw that into a sentence this afternoon. Work, right? I came next one. Is it Welsh, or did I make it up? E gamma o gramme? I see. I think your pronunciation is probably not there with Theo company. Australian isn't isn't helping the Gormogon.

Peter Sueref:   7:56
Um, I'm gonna go with That's a rial. That's a real welch. Yeah, but don't

Jay Stansell:   8:01
you think it means I don't know. Maybe it means that I don't know. I don't know. You got no game on. So if you have had a few Guineas and you're walking home and you're sweating a bit, you're eager Moghaddam really not going in a straight line. I can use that one. Yeah. Yeah, you could use it when talking about trend lines. You know, Theo, for into conversation, that's all for the Welsh s. So you definitely got, Thank God. Well, I think to have four there. That's pretty good. Yeah, Okay, I'll take that. That's going to be some woche linguistic classes, but yeah, we'll get a good All right, let's get stuck into it. Building data products inside a large corporate machine. Now, before we jump into that German sharing with the audience, a bit of your career background and history and your context. Yeah,

Peter Sueref:   8:56
sure. So I've I've worked for San Trigger, which are the kind of owners of British gas on Do you know, one of the largest energy companies in UK for my entire career. So I graduated 2000 to computer science, then went straight into a kind of development role where I was writing code day to day. And then I moved slowly mohr into the data world, things like data architecture in data platforms. Then the latterly may be the last 8 to 10 years. It's been kind of Big Data's that arrived, you know, the early 20 tens and then in two data science. So where I run the first data science? Stephen Centrica. Yeah, my career. So I've Bean a computer science graduate. I'm kind of doing a part time PhD in computer science as well. I've bean on the review for digital innovation in Wales. I'm on the advisory board, a few bodies around kind of education in wells, particularly relating to stammer and data science on technology. So that's really a rather I'm interested in. One of the things that I've been a big advocate for in Centrica is innovation on a particular kind of technological innovation related to a I and data signs. And how do we, um, launch some of these kind of interesting things that we're doing? Can we get them kind of out into the world, little more rather than just keeping them from within our own confines. So that's Bean, That's that. That's the thing I think we might find interesting talking about today because it's it's not always an easy path.

Jay Stansell:   10:24
Now, now, on DH, I can imagine that the fuel you need for data products, these data and Centrica must have an enormous amount of data. Yeah, yeah. So actually,

Peter Sueref:   10:36
our our kind of jump into big data wass as a result of smart metering. So in the UK, we've now kind of adopted smart metering fairly widely. And we've got a mandate from the government to get pretty much every property on a smart metre in the next next couple of years. That meant that you're going from kind of metre reading. We give you two read four times a year, maybe twice a year, maybe once a year, to getting reading every 50 minutes every 30 minutes. So the amount of data suddenly overnight just, you know, absolutely exponentially explodes. So at that point, you think Okay, we need to do something differently. We can't just have our kind of old existing data warehouses we need a big data platform, so that's the kind of initial growth. And then after that I think it's probably is arguable. But I think every big companies a data company, really our data is our most important asset. You know, that's That's the thing that you're leveraging to understand your customers better where your future product direction is, how you market to customers, how you understand the the rest the world on market trends. So, yeah, we've got a tonne of data that we're probably still figuring out how to best leverage all of that,

Jay Stansell:   11:42
right? Also, it's interesting to hear how the dependency on accurate data goes all the way back to the home with this smart, smart metre. Yeah, yeah, irritation in the change there, I do know the storey, my granddaddy spit with Decker and diver in his day right and try to bypass the mirror of these hands. Very common still on. He actually blew himself up and shot across the driveway to the front of it was fine about it. Livingstone a storey. But I don't think you drive

Peter Sueref:   12:15
it again. Probably sensible way have ways and means finding out what happens now using data, actually, write some really interesting algorithms where you can kind of detect

Jay Stansell:   12:25
frost. Yeah, technology. 40. Wow. Fascinating. Okay. All right. So let's talk a little bit around. You got massive amounts of data. I know from being in organisations that do have big data. It could almost be overwhelming where decided. It's just where do you You start with it. And how do you create real value? Means something to revenue growth, for instance, or whatever strategic direction is. Okay. Could you talk to me a little bit around? What? What practises in mechanisms? You're You've learned from having worked. You've come across that he's really working for you and the team. It's intricate. Yeah,

Peter Sueref:   13:06
sure. So And we Yeah, we discovered exactly same thing really early. Actually, we've got this big data platform. We throw all of our data into basic huge hairdo cluster, and then what the hell do you do with it? How do you make sense of it? So historically, you kind of architecture data from the ground up, right? You've kind of got these multiple layers where you surface or your data into these lovely clean off. You know, supposedly clean table's dimensions and facts, and so on on and then your analysts can go and do things with them on that world, I think is pretty much gone. Released change significantly. So we actually took the unusual step of trying to build something ourselves, too. Figure this stuff out. So we had a kind of ongoing project called Project George, which was named after one of our data architects that used to talk about this stuff a lot to try and figure out. How do you best kind of makes sense the data you've got in there. So this project was aimed at Can we create a kind of meta data repositories? We can understand the definitions about data on Also, use a new algorithm to basically find links. Find joins in the data so you can do something with it and understand, You know, the inter relationships on that then became something that we kind of packaged up on, gave to the rest of the organisation on DH. We thought, actually, it is pretty useful. Why don't we go and do something with this? This became our first kind of external product, so we actually launched a product called Io Tahoe, which is part of the Centrica umbrella. Now onto Central umbrella on That's a data company so central, this energy company increasingly going to services we actually offer data is a service. So we've got a data company on this. This project I Tahoe was literally something that we thought. Can we make sense of our own data with this Because, you know, how do you actually kind of understand what you've got sitting underneath the surface,

Jay Stansell:   15:05
going through that process? How did you find solving that problem for you internally Vs crashing into a product that they serve a purpose or a solution for other companies?

Peter Sueref:   15:18
So yeah, entirely different. They were entirely different things. So, hee, I mean the the amount of work it takes to get something built internally. I think it's it's significant, but it's a different type of work to getting something launched externally. So where we were building I mean, you still write the code, you still work in the same kind of method terms you've got bunch of Sprint's you got product owner, you've got this kind of end deliverable. But when you launch externally, then suddenly you have to talk with commercial teams on with legal on DH with regs on with marketing and get kind of financial approval at board level on it becomes a whole different board game and inside a big organisation, you know, were designed to do a couple of things, which is let's do energy less to services for kind of customers. US. Residential customers on business customers were not designed to sell software. You know nothing about our organisation is set up to say, Oh, this is how you go and manage a software products. You sell software. So actually pitching that was quite a difficult things. We had tio internally pitch that very similar to how I imagined you'd have to go on pitch two VC fund to get some, you know, initial investment money. So we we've got this product that we've built. We're using it internally. It's getting some traction on. Then suddenly we go. We have to go and figure out OK, does this? Can we just pick this up and drop this in another organisation? How does this work exactly? S O. They were pitches to various kind of investment boards, internal investment boards, lots of conversations, you know, high level with the boardroom that I've not been invited back to since Because it's Yeah, well, I don't need to go there, and I'm quite quite happy not to go there. John Arena in such a such a big organisation. But I think the great thing was actually Centrica kind of really understood the value of data on DH. They invested in it and then they went and actually made this up. And they want to launch this data company. We've got a few other similar organisations on centric. Oh, so we've got a product called hive, which is the hive active heating products manages your heating and, you know, like, kind of remote control on your phone so you can go and kind of orchestrate your your house space kind of in the home. We've got company called local heroes, which is kind of like uber for field engineers. So where we really appreciate the value of data and actually, data and service is probably the thing that's going to be the the kind of main selling point I think of the next 10 20 years, particularly kind of di carbon isation becomes much bigger force as well.

Jay Stansell:   17:54
Product ising Any data set that belongs typically in turn, into an organisation. I can imagine you come up with a lot of conflict around how we use that data. You have seen a regulated industry as well. How did that impact again? How you brought this this data product or data service to market nine. That you still need to protect you data That's yours or

Peter Sueref:   18:19
your customers. Yeah. No, absolutely. That's absolutely critical. So where we have GDP are in the UK, which is, you know, it really emphasises that we have to protect and respect customers data, and they owned the data. Um, the other thing that we've done, actually which I launched a couple years ago, was an ethics board internal ethics board for data science. I work in data science, and one of the things that we do is build algorithms to do all sorts of things with, you know, potentially customer data or, you know, a techno technological data. Or, you know, whatever data might be So we have to go through our own ethics board to get approval to go and launch something. So we're very firm on the fact that actually way have to pass not just the kind of legal and rake side of things, but is this the right thing to do with customer data? Right. We need to make sure that's that's kind of not just legal, but actually ethical and moral, how we approach it in terms of the data product. I mean, the product we built was really a kind of meta data management to also, it's almost data agnostic and kind of put any data in there, and it doesn't really care what you do. It's just gonna find links and joins in it on. It's really up to then the next layer up of how you go and use that data that would have the impact, you know, in terms of GDP are or legal or regulatory impact

Jay Stansell:   19:39
you mentioned at the start. You This was a proprietary product that that you built yourself a cz, a data leader. The choice we have any type of product is actually build borrow by What was some of the trade offs that stood out to you as to why building it yourself? Yeah, made more sense than all of the tools and technologies. Inventor solutions are out there

Peter Sueref:   20:03
s O. There's still probably debate inside Sandra around exactly around this on, you know, going back at that time, it was not clear Cat which went to do I think of the time The market was very mature in terms of big data. So if we go back to kind of 2012 2013 there were Evander's, but there were no vendors were doing the exact kind of mix what we were looking up or if there were that maybe they didn't fit with our existing architecture. So we we thought, actually, you know, our core competency is energy. That's the thing that we know how to do well and we'd always go build probably first if we're gonna go and do saying energy related. But increasingly, we're seeing that the core committee needs to be data as well. We need to really get fundamental in the study of how to use this data. So I think because of this mics off data becoming more important, more vital as an asset on the fact that they weren't really mature vendor solutions out there. There was some open source pieces, but nothing really that kind of hit the mark we made the kind of decision. Let's go and do it with the kind of decision within it first. Then we kind of pitched up and sold that to everybody else. But, you know, went down well enough that we managed to get further funding and investment and then eventually launched this separate little start up to the back of it. But it's I think that's always the discussion point inside a big organisation. Do you build him by? Do you do something different? Yeah, I think we got lucky with that. One was Well, probably you're always taking a gamble with these things, right? You never quite know how they're gonna pan out. But that one Yeah, on it improved us as an organisation as well. Actually, the process of going through it, I think you know, you become more mature in terms. You understand what is needed to go and launch something out in the world at that point.

Jay Stansell:   21:45
Right? OK, you made it sound very simple. No, no, no. God, no design idea ing. I imagine Iterating to bring ah this new product or service the market. But I know and I'm sure many people listening now as well. First hand that it takes a particular type of culture that happened successfully within a reasonable amount of time as well. Could you talk to me about what is that culture that I would that success to happen? Well, the centric

Peter Sueref:   22:17
is 100 year old company of British gas. At least you know you can trace his lineage back over 100 years. So it's a very big old traditional company. It's not, you know, not historically particularly fast moving. I think up until we launch this product, we'd had one patent application in the last 100 years and then with this product, we think we read another 10. They're all currently being argued about and hopefully will be. Perhaps so we, I think, had to get involved in culture change. Culture transformation is a a big part of delivering this on again, master long going that never ends and because it's such a big organisation. Actually, you kind of target different parts of the order before, you know, rather than the whole thing. So for us, I think we did a few things and we're still doing a few things quite well and other things you could do better. One of things we do is we show our working a lot, so we're constantly do show and tells you every month we're gonna do a show and tell off what we built. What kind of interesting new data signs pieces out there? We also I tried, emphasise. Let's show things that don't work as well as the things that work. Let's show some failures. Let's show that actually, we are in a bit of a guessing game. We're exploring this stuff. It doesn't always take off. And that's being, I think, fundamental and getting by in particularly from kind of senior leadership as well. Because, you know, that starts to spread and it starts to build up a bit of a kind of wellspring of interest. So during that has been great. My team bless them. A brilliant, and they do things like they'll run workshops. They bring your data days so you can come with your business problem. Bring some data and we're going to show you how to code an algorithm and are a python or something. It must be so. They've launched independently as well, which is being, you know, really, really good on these kind of lit the sound like little things. But actually, I think they're more impactful than some of the big kind of corporate campaigns because they come from actual people delivering the work, and they're aimed at the people that are going to kind of get involved in the work. Hopefully. So I think from the ground up certain, we've done a really kind of a good job where maybe it's difficult sometimes is Tio start from the top down. And to get a kind of cultural change, the top times you need to have leaders that have a guess with same desire to do that has you on Biff, you don't have the men. It's kind of difficult to change that. Luckily in Centrica, you know, we've had a few people that have really kind of believed in data in transforming the organisation in innovation. So over the past five or six years, we set up a whole party, opens a trickle century innovations, which is kind of an internal VC fund. To do things like this, you invest in potential external startups but actually also invest in internal startups as well. So innovation competitions and you know all these great things that you see companies doing. We're lucky enough to have that as well, which is being really helpful,

Jay Stansell:   25:11
right? Can I ask what we've company this big? And with fans dedicated to internal and acquiring angst own external ventures, you almost too big to foul, You know, when there are failures. What what happens with that? This year you received Because it's a public company? Uh,

Peter Sueref:   25:32
it's It's so it's Yeah, it's on. It's on the 4100 company. Yeah, yeah, yeah. So, yeah, that's a really good question. We So we haven't had a big failure yet. I guess it's fair to say probably will. At some point. I know. You know, some of those started Betsy made maybe having paid off, but, you know, that's the nature of this thing. I guess I could talk about some of the smaller pieces that we've done so with, with with my department with data signs, you know, we're trying to launch a little products first, then maybe might turn into something bigger on DH. Some of those take off some of them's don't. What we tend to find is that actually find lots of interest from our business users to Can you go and explore this? Me? Can you go and build something for me on down when it comes time to production? Isaac. Actually, we haven't got the money right now. We might get budget next year, maybe come and talk to us then on. Do you know that's kind of work that's being wasted or what has been lost on that happens fairly frequently. That's, you know, that's not an uncommon occurrence, that kind of a source of frustration for me, my team. But understandable, I think. Because until you see some of the output from these things, particularly if we talked about kind of a machine learning, then you need to build up a lot of data. You need to get some historical data in there and start to kind of get the training improved, and then the outcome improved. And it's understandable that the business is going to be slightly wary that so we tend to see lots of those kind of Peter out before they re tradition. But at that point, we haven't spent huge amounts of money that's being kind of almost are indeed many exploratory money. A lot of this is by nature are Andy. What would happen if a big part failed? I don't know. I guess you'd be seeing our share price. That would probably be failure back full. I don't think Central Too big to fail, though. I don't think

Jay Stansell:   27:23
we've got a lecture being

Peter Sueref:   27:24
a bank or a big insurance company. Well, yeah. Keep health future. Ls yes, Yeah, yeah, Exactly. It keeps you keeps you kind of honest. I think

Jay Stansell:   27:32
you mentioned there about receiving enquiries from around the business. Small, small pieces of work on DH. I know data teams by nature are very hypotheses. Lead. How do you go about prioritising water? The good hypotheses that well, rationalised versus what's Justin idea. And someone's just got a question for nothing more than them personally had you make that sort of almost product management prioritisation decision around where you invest You talk about

Peter Sueref:   28:03
it a lot. Thiss again is something that my leadership team talk about in team meetings. How do we prioritise the work? Because it's really interesting. It's really difficult to figure out which is gonna land on witches and Andi. Actually, I think they're probably two ways that I've started to do almost unconsciously, my head. When is have we got a kind of really passionate product owner on the other side? The really cares about it, and you can tell they're invested in and they want to do something, so the question might not be great. But if the person is enthusiastic and you can see they kind of want to see this thing through the matter rial kind of green flag for me, you know, it's a red flag. If they're kind of, you know, turned up in Well, I've heard about date signs. You know, you might want to do this with you. What do you think? That I'm probably not gonna be as keen, um, the other, the other kind of crash. And I think I saw this summer on Twitter. You know, try and credit at some point if I can, but somebody said you should have If you started data science problem, what is the thing you'll change in the business as a result of this? That should be your first question. And if they can answer that question and that's also another green flag. So if you know that I'm going to, you know, build this algorithm for you on. Then you will change the way you handle customer calls in this way, or you will create a new process to do this. Or you will do something that's that's great because you know it's gonna be impactful rather than again. Can you explore this data and tell us what it means? You know, that doesn't tend to give good outcomes or not. Not impactful outcomes. Anyway, it might be the start of something, and we still do some of those projects. But you know, you're not quite sure.

Jay Stansell:   29:38
Could you talk to me? It's going to the world of that Sounds a little bit deeper. Now I'm getting out out of Mountain Dew from the show. So how do you go about it? A Riding on an algorithm in an efficient way.

Peter Sueref:   29:50
Oh, that's good. Yeah, that's a good question. I mean it. Sometimes it happens like that, but most often so we've had We've had an example where we've built a product for our HR team, which we're trying to product eyes. The moment on DH without kind of going into the detail about the product is about. We've actually found that we've tried to improve this product not by kind of iterating on the existing album, but by adding Mohr on different algorithms to do it. So we're trying to kind of figure out a certain thing. Andi use a particular method to do that. And then we augment that with other methods as well. On top of it. So we keep adding and adding and adding, and actually that environment is the best way of bringing the accuracy. It really depends on the domain. Your you're involved in a swell. So if you're involved in a where you're trying to maybe classify something and you've got very few cases that are positive, then it becomes quite difficult to do, and you need to find out mohr and different ways of doing it. Whereas if you've got a fairly balanced data set where you're you're kind of classifying things into almost to equal camps and actually, yeah, you can you can kind of fighter out and you could improve the algorithm consistently. Or maybe tried different algorithms that might change over time. One of the standard things you do is you have a champion challenger, so every period you know, whether that's a month or 1/4 a year, then you'll update your algorithm with, you know, new training data on DH. Maybe you'd change the YouTube. Now you change the parameters or look at different algorithms. So you constantly you're constantly looking because the world isn't static, The world keeps changing. So yeah, this No, it's not a single answer for that. I guess there's a lot of different ways depending on the domain, you're involved in

Jay Stansell:   31:34
a lot of the science hypotheses. When I worked in fortunes, work down scientist has been very exploratory. Long of the Siamese A lot for me. It's a bit like prospecting for gold. Yeah, you know, how do you How do you time box at with your team? How do you make sure, you know, just you have everywhere forever on and put a limit on things. Well, so we Yeah, lots of lots of the

Peter Sueref:   32:00
date scientists. We have come for academia where the time skills are different. They just come out of PhDs elastic kind of 3 to 5 years, and then suddenly they're in a commercial environment where we've got 3 to 5 weeks to do something quickly on that. That takes a bit of a mind shift. So the things we tend to do, we'll use kind of agile methods. So, you know, I think we use our job pretty much across the board you some Cambon somewhere. But this, I guess, focuses you on having outputs on a regular basis on DH. Then we've got product owner attached to pretty much every single project, and it's the product over. They get to call the shots around you. Does this look like it's going anywhere we lead into any output? S o, I think having the combination off an external person, you know, exchanging a PhD supervisor for a product only, I guess, in some ways. But, you know, it kind of works. Having this external person having these very short time box activities have kind of 1 to 2 weeks or three weeks. In some cases, that tends to focus on the kind of output there probably isn't a hard and fast rule about when to kill a project. Though we're probably not very good at killing projects. So I'd like to see a few more kind of killed in there very early infancy if it doesn't look like they're going anywhere. Um, saying that some projects tend to bear fruit match later on on DH. One of the things that we do is give our people some free time to work on things they think is interesting. I think you're interesting. So we've got a few projects that ongoing. They're kind of like it's almost like the Google Time thing where I think they give 20% time we give we give some time kind of like half a day to go and work on. Oh, I think this social network problem is really interesting, but nobody wants to fund it. So I'm gonna go have a look and then maybe if it turns into something, we can go and launch it, or somebody else built this really great kind of computer vision piece to go and look at solar panels on Google maps to go and figure out where all the solar panels in the UK so because there is no central register of solar panels, is independent companies rise things. So how do you figure out where solar is? So there wasn't really a project that few people interested in it or would have been interested if they had that. So this was a spare time project that's gonna build a deep learning model to go and figure out solar panels. So things like that tend to be interesting. And then hopefully they become useful as genuine kind of, you know, things that provide some value.

Jay Stansell:   34:21
Awesome. I mean, big, big data typically for me assumes you have all the data. Is that true? No, no, God, no. God. So I mean again,

Peter Sueref:   34:30
it depends on the domain you're talking about. So we've got I mean, let's say let's talk. Let's talk about customer data isn't something we do, And I haven't explored the GPR implications or anything. That's a purely hypothetical before anybody

Jay Stansell:   34:43
wants to see me but say

Peter Sueref:   34:44
we've got tens of 1000 calls coming in every single day, So potentially us an awful lot of voice data going. We could potentially do sound like turn that voice to text or do something like your kind of sentiment analysis based on the tone of voice and Hamburg and the loudness and all these other features. Eso Technically, the data's there actually to go and mind that data, you need to go build this whole pipeline's whole process behind it, which is gonna be expensive and time. It'll take a lot of time. It will take a lot of resource that need to be demanding. Where we gonna get a the end of it, Do we Nose can provide value. So again, you're kind of you're always making bets on these things is never quite clear cut where you're gonna get the value. Something like that. No doubt Google and other big organisations of thinking about how to do that. Microsoft, you know, how would you build a pipeline to go and do that automatically? It's not something we'd we'd go and build ourselves because again, it's not our core competency, but, yeah, that data's technically there right now, so we could go and use it. But you should know whether he should and whether you want to invest the time money.

Jay Stansell:   35:52
Just a strategy set those guide rows quite strongly for you from innovation perspective. Guys don't don't go into this space and don't go to that space. Or is that down to your own s? Oh, yeah.

Peter Sueref:   36:05
We've got a kind of various strategies. We got data strategy. We've got on overall organisational strategy and ahs you can imagine that tends to fit into if it's your core competency, then kind of go and go and do something they're gonna build. If it's not, then we'll buy something you could have it in. And I usually has to have some kind of ownership from the wider business. Um, but in terms of if we're looking out principally exploring something that actually knows very loose that we do have we have a kind of a very research focused team that are able to go and explore some of these things and understand the value of them on. Then pitch back up to strategy in architecture and say, Do you know what we've explored? This? This might be a good thing for us to think about more widely, so it tends to be a kind of ah, good kind of mutually beneficial relationship, I think.

Jay Stansell:   36:52
Right. Okay. The number of PhD grad's coming out now. We're with that science missed. Its name is awesome. Onda, obviously the salaries associative that makes it very attractive for people coming from masters up into paged in data science. But for those that have not like yourself that's doing a PhD alongside career but have taken a purely academic start. That's how come from on undergraduate to the Masters. To PhD. Yeah, and in coming to a commercial environment, Yeah, what are some of the common things that they do that you'd recommend? Don't Dio could make things easier for them when working with product teams after coming out of an academic. And,

Peter Sueref:   37:36
yeah, so I think the thing that shocks most of the third kind of new started in academia is you're in a commercial environment environment. Now there's a focus on Where's the money? At the end of this, you know your time costs money, and the output from your project needs to have a financial value attached to it. On that is, I mean, yes, I was fairly obvious that something to be working in a commission by over 20 years, But for some of our new starters, then that may be sometimes get lost. I think having an appreciation of how the business works is probably really important thing because it's completely different academia. They're completely different goals and approaches, maybe less of now in the UK actually over, You know I think we're talking academic, currently on strike. Actually, a few these issues and they, you know they may be out became more commercial. But you know, it's it's the whole framing for the project you're doing is different on understanding a commercial environment I think is critical. There's also probably a little bit around kind of standing on your own two feet, a bit more so in a PhD. I mean, you have to do that. You have to do independent research at the same time you got a supervisor there. That's kind of giving you advice. And you got your kind of academic peers and colleagues. And to some degree, you kind of fall off a cliff When you first joined a big organisation, you know, you've got colleagues and you got it, boss. But you're also expanded to be expected to be very independent. And if your data scientist well, you know there are certain expectations of you needs to be able to go and do independent, high quality work that has some kind of outcome in the organisation of commercial outcome. So I think that this kind of focus more on the commercial side on focus on your kind of independence. Probably critical on the final thing. Actually, this is probably most important thing being able to express yourself and explain your ideas really clearly in a commercial environment, which is really different. So academic environment again, They're talking to senior leaders. You don't want to talk about hyper parameter tuning or, you know, distributions you need to talk to about again. What's it mean? In terms of commercial value on DH? Explain things in clear language. That's absolutely critical is probably my 1st 1 actually.

Jay Stansell:   39:45
Yeah. Yeah, I should've said that. Right Beginning. Yeah. Um, awesome. And for PhD, grad, sir, have lost my train of thought there for them. Them coming in and working in that environment is peer review off what they're trying to operationalise. Still critical. Like it is in the academic world.

Peter Sueref:   40:07
Yes. So we do peer review way we tend to weigh, try to help projects with multiple people on them so that we have the ability to kind of share code and work on code. Together we peer review, particularly if it's gonna be a production code that gets peer reviewed on DH. That gets period kind of code quality as well as your kind of assumptions. And, you know, you're basing this thing on sound logic and science. Um, it's a different kind of peer review, I guess, to academic because, you know, we don't have papers that we have just sharing publicly. Yeah, Yeah, we're all sharing a publicly, so it has a slightly different aspect. We wanna make sure that this code works and does what's expected to do on, Of course, in a commercial moment, you've got testing. So everything has tested multiple times, you know, integration, testing, ration testing and so on. Um, so, yeah, hopefully, by the time the code actually hits the world that it's gone through. Is this sensible? Does it kind of do what we expected to do? And does it break down on, you know, should should have gone through all those steps. We do some pear programming as well, kind of, you know, encourages kind of instant feedback. And instant resolution is the stuff that works quite well,

Jay Stansell:   41:21
brilliant for product managers or product leads that we're gonna be working with that Science teams for the first time. What has some of the ways of approaching a data science team to maximise the value from them, that you'd recommend toe product people product managers.

Peter Sueref:   41:37
Yeah, So again, lots, lots of camp academia They are not. All of them have. But there's definitely a kind of approach with data. Scientists who say they care really about the hypothesis there trying to test me care about the the underlying problem. So getting really clear on the problem you're trying to solve and what outcome that will have. I think that's the really critical thing with getting a good outcome with data scientists. So if I turn up with a product idea, Andi, I kind of pick a random example. I want Teo optimise this chair. I'm sitting on it. It's really uncomfortable. How do I How do I go and do that? I mean, I could either say, Well, I think a wheelchair is a bad and I need something completely different going. Think me something up. Data science is going optimise the whole world, or I can say, actually, we limited we've got we know we can alter the length of the legs on the cushion campiness. Can you optimise for those two things? So the second, the 2nd 1 will have a much greater kind of ability for our date scientist to come do something because they got kind of boundaries they know we're trying to do. They know the outcome of the problem. The 1st 1 is a five year research project in a PhD. Right. And then you're gonna get some papers at the end of it. You know, like my thesis, but probably not a commotion environment.

Jay Stansell:   42:55
Strong hypotheses set the constraints, try and give them some direction. Yeah, exactly. Not just the only direct on DH. Similarly, how would you suggest engineering functions? Who? Maybe that science is being introduced to the business. How? What would you recommend for engineers, Teo? Build a relationship with data science team? Yeah. So this is this is

Peter Sueref:   43:18
again is really I think it's still an ongoing discussion. Probably act in the world around kind of data engineers, data scientist, Deb, ops teams. Um it feels like it's critical to get all of those people in a project as early as possible together. So why don't think works is if you start up with the data scientists and then we're gonna throw some code over the fence of data engineers. Then can you go and implement this? Please come plug it in somewhere. And then that turns into a dab ops process further down the line. It feels like actually to get a really good outcome. You want to get all those people into the project, right? For the beginnings, they all understand the problem on DH. The data engineers can be just as involved in fixing that building the product as a data scientists. So in Centrica, we've got data engineering teams. They work kind of working squad to be kind of used this kind of Spotify squad model that's being, you know, all the rage for wild. We call it called a different things essentially kind of matrix working. So we get all these people in together early on, build some report, build some kind of, you know, actual understand the people that you're working with to some degree, that's, you know, hugely important because there are people at the end of a religious resources on DH. Then the data engineers were finding a contributing toe how the problem gets solved. So it's actually, you know, we could go and do it here because this platform supports this particular thing rather than you know you're building all this in our actually, you've got this kind of bison pipeline. Here's where you go and do it there. So so actually getting them together, I think really, really early on. And they're all technical people. They all, you know, it doesn't It doesn't tend to be, I think, is greater divide between data people as that does, sometimes between data people and kind of commercial people. I think that's the thing. That's that's probably the thing that they fall down if anything does.

Jay Stansell:   45:08
Right. Brilliant. Thanks so much for this session. Peter. It's been awesome. Just for pleasure. Thank you for having me. I've certainly plenty. And your first date scientist I've had on and I know I could go for another five different topics. Eso has been an absolute pleasure. Thank you very much. Thank you. Brilliant. If you've enjoyed this episode or any of the episodes from the product coalition European to please remember that I'm doing spending this time to raise awareness and funds for the Bush fire affected communities Australia as well as the wildlife volunteer firefighters. You can show your support at Bushfire dot product coalition dot com until the next episode. Thank you very much and thank you all for listening speaking.

Introduction
Cardiff Quiz
Career background
Building data products inside a large corporate machine
Practises and mechanisms of working with Data
Solving problem internally VS Creating external product solutions
Usage of the Data and it's protection
Why build the product internally
The culture that enables the success
Failures in a big company
Prioritising of the time investment
Iterating on an algorithm in an efficient way
How to time box
Strategy and innovation
How to adjust to commercial environment after coming out from academic environment
Peer review - is it still critical?
How to approach Data Science for product people
Engineers and building relationship with data science