The Ethics of AI in Product Management

From one of the foremost leaders in the machine learning field, Ammar Jawad breaks down his thoughts on topics like psychology in , machine learning in the tech industry, and the ethics of artificial intelligence. With experience ranging from marketing to creating AI strategy, Ammar offers a fresh look into some of the most fascinating aspects of the product world.

Question [00:06:50] you started working at a very young age in different positions in Denmark as a sales assistant (even as a telemarketer), and of course you’re in a completely different sphere right now. Do you think any of these experiences that you had at a young age helped you in getting to where you are right now?

Ammar [00:07:26] So I’m just going to share a story.

I studied marketing at university in Denmark, and I really despised it. I worked at Microsoft, both as an intern and also part-time afterwards. Marketing taught me a lot about the intersection between product, marketing, business, and psychology. That is often what marketing focuses a lot. So ideas like, ‘how do we sell a product’ and things like that.

That was very helpful, but I got very annoyed with marketing when we started applying it at Microsoft because I had to depend on my manager and their vision for what we should be doing. And I wasn’t literate in data and statistics, but I knew there must be a better way than this hippo approach with the highestpaid person’s opinion being the only one that mattered.

That’s why I started getting into statistics. To give an example, if you’ve never been sad in your life, then it’s really difficult to appreciate the good times. It’s similar here: if you’ve never experienced what really doesn’t work, then it’s very difficult for you to actually come up with ideas for how things should work. So my transition from marketing into business analysis and then senior data analyst showed me what I felt was lacking in marketing.

Then very briefly, I worked as a marketing data scientist, and that’s where I started to combine that full scope. Now at Expedia, my scope is huge; it’s working with machine learning, teaching the product teams how to build products for machine learning, as well as the machine learning deployment capabilities. I’m teaching employees across the board everything to do with machine learning.

I’m the product lead on machine learning here, and I’m very thankful for actually having to go through that painful process of being annoyed with marketing. Now I actually have a better understanding of what I should avoid and how to actually build something that does work.

Question [00:19:02] So I wanted to ask you about your experience with machine learning, and managing massive amounts of data. What do you think about the future of machine learning and artificial intelligence? And the role that PMs play in programming certain behaviors. What is the practical side of this?

Ammar [00:20:12] That’s a very good question.

One of the immediate impacts of machine learning adoption being accelerated across companies is having fewer bugs. The reason why you’ll have fewer bugs is becauseyou have software engineers that write code to achieve a particular outcome.

For example, think about a calculator: every time you enter two plus two, it will equal four. In machine learning, you would do it differently. You would say every time I give you these input — like two plus two — and you should observe this desired output, which is four. So you write the program in the middle; you figure out what the function should be to actually get to that desired output. What that means is correlations to find out what the best program is to write that software, or to write that program.

And again this is a consequence of having fewer bugs because as you’re starting to build more complex software with different types of developers involved, the likelihood for bugs being introduced to the system increases. Now that’s maybe a minor thing, but also worth taking into account.

The other area, I would say, which has been my own motivation for studying machine learning is job displacement. I’ve spent two years in my life — after my associate degree in marketing — being unemployed and I couldn’t stand that feeling. It felt so humiliating to be unemployed (at least in Denmark). I decided I never wanted to be unemployed again, and that’s why I actually upskilled and have taken 50 online courses over the past five years in machine learning, statistics, and coding. So job displacements are going to be very real and they are going to be much more comprehensive than we had initially thought. This is because the type of tasks that machines can actually automate are very diverse.

So if you were to think of the steam engine, it displaced some jobs but that was more manual labor and some factories. So they displaced some jobs but created different types of jobs as well. Again, to ultimately operate these machines that are machine learning is going to be a bit different because it’s not going to be low skilled jobs.

Actually, it will create more high-level and high-skilled jobs. One example is if you look at a radiologist, what they have to do is look at an image and then make a diagnose and say, ‘all right, this requires further analysis by a doctor‘ or something that can be done by a computer vision model detection of cancer.

Actually, the BS that I studied with, they did a project for cancer detection and they beat the Stanford University Board of Medical Directors in their accuracy of detecting cancer. I believe it was 68% accurate in melanoma. So now you’re seeing these are very complex areas, but it means that the type of displacements we are going to be very diverse. It’s not just going to be manual labor, and that’s a bit scary because it means anything that is pertaining to perception could be automated.

But again, put it in context because it’s more these tasks, not the actual work, so I think people need to get into a completely different mindset. We need to start forming these study groups, or local Meetups where people come together to study the skillset of the future.

Question [00:25:04] how have you worked to establish relationships with stakeholders within the company? How do you go about that, and what’s it like having the role of ‘defender of your product’ and champion of your team?

Ammar [00:25:26] Within the product organization, my responsibility is machine learning adoption. Which means the number of machine learning use cases that are being adopted, and also how many iterations of preexisting machine learning features that we work on every quarter.

My job was really to accelerate machine learning across the company. This is a big learning piece, for me as well as for the different product teams. This means you have to be able to look at any page — for example, booking hotels — I need to have the knowledge to say, ‘Alright, on this product details page, you can be using machine learning for this, for that, but not here.’ So I need to have a very clear understanding of where machine learning can be applied and where it can’t be.

To give you an example, if we were to look at security — like providing payments — for people that are booking, as they click, they are expecting to be able to pay knowing that the connection and that we are handling that information very well.

That’s not a machine learning problem, right? Because this is about ensuring that consistency. We don’t want a user in Ghana to have a different experience than a user in London when they are booking here. We need to provide every single user with the same highquality security when they are paying and providing their payment information.

But as you start to look at areas where you have to see if certain types of users find this more relevant than others, that’s when you enter a machine learning problem. And we have to ask ourselves if we interested in the averaging?

That being said, averaging is really cool. It is the concept of saying (for example), ‘We launched an ab test, and on average the best variant was this (hypothetical) one’. It could mean that you are pushing a feature that makes the experience worse for 40% of your users, but just because the majority actually trump the minority in this case, you’re going to be pushing it live.

So machine learning is dictating that this is not acceptable in 2019. What we need to be doing is looking for what works for certain users. So this is the concept of personalization, and really I hate that term even though I have it in my job title. What we really should be talking about is relevance, right? So each persona, each customer segment, should see the feature that they, will interact with more favorably. That’s all possible through machine learning.

That’s a wrap on this week’s episode of The Product Podcast!

Open another dimension of technology with our next guest, Andrea Chesligh from Boxed. If you have your own thoughts on the topics covered, then drop us a line on Twitter — we want to hear from you!

Don’t forget to join us next week for fresh insights.