Why Your Free Users Are Your Real Growth Engine – Cem Kansu, Duolingo CPO

David Barnard:

Welcome to the Sub Club Podcast, a show dedicated to the best practices for building and growing app businesses. We sit down with the entrepreneurs, investors, and builders behind the most successful apps in the world to learn from their successes and failures.

Sub Club is brought to you by RevenueCat. Thousands of the world's best apps trust RevenueCat to power in-app purchases, manage customers, and grow revenue across iOS, Android, and the web. You can learn more at revenuecat.com. Let's get into the show.

Hello, I'm your host, David Barnard. My guest today is Cem Kansu, chief product officer at Duolingo. On the podcast, I talk with Cem about the premium trap many apps fall into, why free trials work, even for freemium products, and how a try for $0 and zero cents actually outperforms try for free.

Hey, Cem, thanks so much for joining me on the podcast today.

Cem Kansu:

Happy to be here. Excited to chat.

David Barnard:

So I wanted to talk through Duolingo's monetization story. I mean, a few apps out there are going to have tens of millions of users and be in the position Duolingo was in when you started monetizing, but I think there's so much to learn from the lessons y'all learned along the way.

So let's talk about that. Duolingo was a free app, famously, completely free. And then the first step was layering on ads.

Cem Kansu:

It was actually was a website. It was not a mobile app. And this website's idea was actually to be completely free. So there would be actually no monetization for the users that are using Duolingo to learn a language.

However, as users did exercises to learn a language, which includes a lot of translation. So if you're learning Spanish, a lot of your exercises, you're translating Spanish to English or English to Spanish.

Users would do these exercises to learn, but there would be a crowdsource model to collect those translations and eventually turn that into a translation service. We actually did this, this was built.

David Barnard:

Okay. So I heard this story, but I didn't know you actually built it.

Cem Kansu:

This was built. This went live.

David Barnard:

Wow.

Cem Kansu:

And it actually generated some revenue, even though very small. So we would have customers that are, at the time, the big news outlets as well, like CNN, BuzzFeed, they would feed an article they wanted to translate from English to Spanish.

And then our users in bits and pieces would translate that article and we would crowdsource a translated article and then provide a translated article back to CNN. It was actually a genius model. If you want to do something crowdsourced, it's actually, you're serving both sides a positive value.

The challenge with that model was though is twofold. One of them is translation. Also, this is pre LLMs. Translation now has changed a lot post LLMs, but this is, you have to remember this is 2013 as well. Even then, translation was a race to the bottom business model.

So if we were providing a service for, I don't know, however three cents, 10 cents, $3, whatever it is, there was always somebody else trying to do it for a little cheaper by having someone in the Philippines translate it for cheaper. So there's always a race to the bottom. It is hard to scale for that reason.

And then number two is more fundamental, which is the translation quality of a language learner is actually not great. So you end up putting practices in place to up-level the quality of that translation, which then you turn into a basically a translation business and then really start shifting your resources towards basically translation and not teaching.

So we realized that it had these problems. And around the time I joined Duolingo, which was 2016, we had pivoted away from that model and was trying to figure out what the next model should be. Actually, our very, very first experiment with anything monetization was the streak repair as an in-app purchase.

David Barnard:

Okay.

Cem Kansu:

And around the same time the ads experiment started. But I think the first thing that users saw at Duolingo as a thing they could buy was repairing your streak for I think a dollar or $3. That was our first entry.

Honestly, it was a good experiment. We actually learned a lot. We learned that users really cared about their streak, and a good percentage of our users are willing to pay a dollar or two to repair their broken streak.

But then I think then we got into this experiment mode of what really works with our audience and how do we make sure that we keep the product highly retentive and a great user experience? Coming back to your question, that is the moment where we started trying ads. So that was 2016.

It's funny, if you launched, if you go watch, and I think these talks are still out there, our co-founder, CEO, Luis, gave this talks for Duolingo's launch and he said, "We will never have ads, never in-app purchases and never subscriptions." And those are the exact three things that I built in that order basically.

And the reason he was so adamant in saying those was Duolingo's mission is basically centered around providing access to education. And at the time, every education business model, whatever they did, they would lock the education value behind a paywall, get someone to pay.

Think about the Rosetta Stone model. For example, that was the very standard model at the time where you're selling content. We never wanted to be that, but I think we realized there's actually a way to do that, which is provide access to education, but also monetize the bells and whistles.

That's why we set on this freemium journey. Ads fits into that really nicely. If done well, users don't have to pay you anything, but you can actually generate some revenue. So ads was that test.

Honestly, testing ads wasn't rocket science. It was fairly easy to test our way into, of course, being sure the user experience is great, but also how do you maximize revenue from ads while making sure your retention remains awesome.

The hardest part I would say was the cultural shift of going from a pre-monetization app to a post-monetization app because until then, so until the time I joined, there was 60-ish employees.

I think a lot of folks joined the company thinking it was maybe even a nonprofit mission because the mission is so grand and so awesome and the company never quite bothered to think about revenue and monetization very much.

It was a mind shift to the company to be, "Hey, there's things we have to do so we can generate revenue and if we generate revenue, we can actually fund our mission so that we can be more impactful," was probably the hardest part of turning on ads on Duolingo.

David Barnard:

And I guess, you weren't there at this time, but I did hear there's a story out there around a venture capitalist pushing this direction. What's the story there?

Cem Kansu:

So this story I think is when CapitalG invested in Duolingo. Layla, who was also on our board after that investment for a while, she told Luis at the time, "This is probably the last investor you will have in this company before you figure out monetization." So in a rounded way to say, "You have to figure out monetization."

As a growing company, you at some point have to do that. But I think also the common wisdom at the time for consumer companies like Facebook was just grow audience, you'll figure out monetization later. So I think Duolingo followed that, but until a point where I think CapitalG and Layla had this big push to be, "Okay, it's time."

And I guess, that was around 2016. So when I joined, the mindset shift had moved to and Luis's first thing he told me was like, "We hired you. Your sole job is to figure out how to monetize Duolingo." And that's what I tried to figure out.

David Barnard:

That's awesome. What a great story. So ads, they're tricky. How did you introduce ads? One in a way that didn't totally piss off users and tank retention. And I mean, I'm sure there was user pushback and bad reviews in the app store.

This is part of the story, I think, a lot of folks can learn from because any monetization change you make will spur some level of bad reviews or negative feedback or churn. So how did you balance that as you started to roll ads into the product?

Cem Kansu:

This is a very good question. I think that there's the... I'll give the obvious answers and maybe the harder to navigate parts.

First, for a consumer product, you should obviously dog food your product heavily, so that whatever you're doing to the experience, you have firsthand experience on how it feels. Meaning you can easily say, "Oh, we'll put an app open ad."

And if you're just watching metrics, I'm sure that might look fine, but when you test it, you're like, "What the hell is this?" Then you should know what you've introduced and obviously avoid the things that you wouldn't want to see as a user.

Hence, our first test, and honestly even today, that's our only ad placement, is after you've completed a lesson, we show you an ad. Not during, not before, but always after, because we believe that doesn't interrupt your learning.

So one is the qualitative. It should be a good user experience, and you should know that by testing the app yourself every day. The second one is quantitative. Measure everything you can, like is retention fine, is app closes fine? What's happening to your reviews?

You might have to make some trade-offs. Ideally you wouldn't, but realistically, if you go from zero ads to some amount of ads, there is some risk that some curves might come down, but I think you should know what that cost is.

Anyway, that's through AB testing, that's how we measured it to be... Honestly, our retention was maybe a slight hit, but mostly neutral for most of our ad tests over time. That's one.

The third one is there is, of course, you go from a free product to selling something, users won't always say thank you for doing that. Imagine if Uber started completely free and one day they said, "Okay, now we're going to charge for our Ubers." Users would hate that too.

And that's what we did. I mean, in a way, we provided a free service and we kept a lot of the elements free, but we said, "Hey, actually you can buy a streak repair now and you might have to see an ad at the end of your session."

We got some complaints. The thing that helped us have clarity was, one, retention was fine. Retention was neutral on most of these experiments. And when you looked at our community's complaint volume, if you quantify that, for example, per a million DAUs, how many ad complaints are there or add negative reviews, however you want to count that, versus other topics?

For example, I realized I did this quantitative analysis in 2016 because we were at this pivotal moment of like, "Do we launch ads or do we not launch ads?" And we looked at, for example, the number of complaints about why we didn't have the finished language on Duolingo was three extra number of people that are complaining about Duolingo having ads.

So that starts putting it into perspective of it's not zero complaints, but actually in the grand scheme of things, this is not going to break the app and our community is not going to run away tomorrow.

So I think understanding the volume of what you're doing to the community as much as you can gives you perspective. Otherwise, every negative comment as a product builder hits home really hard. Everything looks like the world is falling apart. That's why it helps to look at the broad picture of all of our issues as a product and all products have many issues.

David Barnard:

Yeah. That's exactly what I was going to say is that it's so hard as a builder-

Cem Kansu:

It's really hard.

David Barnard:

... to get those few negative reviews. And I think that's a really helpful insight to look at that bigger picture and understand.

I got a bunch of negative reviews from my weather app for the audacity of charging $40 a year for a weather app. And I did get probably overfocused on those negative reviews. But then when you look at all the people who were subscribing, were staying subscribed, actually I have pretty good retention.

It's like you got to zoom out and not let that small subset of users who are going to complain tank what's ultimately going to help drive the business forward.

How did you work on ad quality? I believe you implemented AdMob first time. And again, in my weather app, I had AdMob. Eventually I ended up pulling it because it was gambling ads with like 777 and animations and like all this crap.

How did you work with AdMob and how do you think about ad quality as part of the experience and how it can actually be detrimental to the experience?

Cem Kansu:

Yeah. I mean, ad quality, I would say we are incredibly strict on, but with programmatic, you'll of course have some amount of control, not full control.

But with anything that is, honestly, it could be even slightly dicey. We blocked all those categories, like if it's gambling, even if it's games that look kind of sketchy, pick your category. Any of those that we didn't feel was also young audience safe or just brand safe, we blocked.

So the starting point is we used AdMob first and then we integrated other sources like Facebook ads or at the time Flurry and now we use also I think multiple networks. We use Unity.

That was to mainly broaden demand, but for category and what kind of ads we allow, we tried our best to block off any categories that are dicey, even though it might be a revenue loss, we were like, "We don't care." The more important thing is good user experience and ad quality.

So we did a lot of category blocking and if we needed to do custom blocking, we went and tried to work with ad networks to be like, "This type of ad has three X buttons to exit. We don't want this in our app. Please don't give us this thing." So we also did some of that back and forth as well. That's our stance.

This year also to improve ad quality, we're also doing our own direct ads more and more because then we control quality entirely. It's an advertiser that we know, we shook hands with, and the format is what we've created. So it's beautiful and it's a great advertiser.

So one now new lever for us is to reduce programmatic volume and increase direct volume because it's just a better user experience.

David Barnard:

Yeah. I love what y'all are doing. I mean, it's hard for smaller apps. You don't have scale to do this kind of thing, but tell me more about how these partnerships work to improve that ad quality so that it does improve the user experience.

Because y'all are going to be incorporating Duolingo characters into the ads with the brands you're partnering with, right?

Cem Kansu:

Have you seen them?

David Barnard:

I haven't seen it yet-

Cem Kansu:

It's very cool.

David Barnard:

... but I read about them.

Cem Kansu:

It's very cool. It's currently only in the US. So we're focusing on growing these in the US because obviously we created a great format, but to be able to scale this, we also need to be working with companies. And right now we're just focused on companies that would want to advertise with us in the US.

We created multiple formats, but the one I love the most, and that seemed to be working really well, is one of our characters. So we have eight Duolingo characters on Duolingo that you see, introduces the ad.

For example, pick a brand. Let's say it's a detergent. I'm making this up. It's not like we had exactly this, but one of our characters, Lily, would make jokes about like, "Oh, I need to wash my clothes. I wish I could do that."

And we write a custom script to introduce that ad that's very short and then the ad itself plays. So that at least we are building a little bit of our character world further and it's not jarring as you ended a lesson right now. That's what our ads are really a little bit where you see an ad, it feels like you're easing into the ad.

We're trying this right now. When you see it as a user, it's actually as an ad, it's a very delightful format. Our challenge I would say is scaling it. I think since it's a custom format, well, you got to go up to advertise and explain how this works.

So it's a bit of a slow adoption process, but we're going to, I mean, our goal is to basically increase the volume of these ads and reduce the programmatic by a good amount in 2026.

David Barnard:

One of the ads you do often show is for the subscription itself. How do you balance, from an LTV perspective and monetization perspective, how do you balance when to show those ads for the subscription and then when to show programmatic or these new brand ads?

Cem Kansu:

So we, just to set the stage, if you do a Duolingo lesson, and this includes any of our lessons, so you could do language, math, music, and chess. At the end of that lesson, you see an ad.

And we decide, like you said, if we should show you a programmatic ad versus one of our internal ads, which could be an ad for a subscription, or it could also be an ad for our other products like the Duolingo English test. But we call those basically internal ads that we decide we've produced and it is a Duolingo product.

Majority of them are going to be either Super Duolingo ads or Duolingo Max ads. The decision logic is mostly now controlled by ML actually, but the ML uses various signals that is trying to figure out LTV maximization at its core and it is looking at every signal that we feed it that could be an indicator for someone that is more likely to buy Super.

Or it also balances out, of course, if, for example, it's shown you the same, or it doesn't really show you the same Super ad three times, but it's shown you Super ad three times that clearly you're not interested in, it backs off and shows you other ads.

So it is basically built to optimize LTV with a likelihood to purchase being the objective function.

David Barnard:

A lot of apps that have ads in them don't do this kind of thing, but as cheap as the CPMs are these days, I think all developers should be experimenting with putting ads in for the premium product.

It's a perfect way to balance that monetization of if you have a freemium product and you're not making that much on ads, which you just don't. And actually, I'll follow up with a question there, but it's a great opportunity to remind people, "Hey, there is a paid product. There are features you can get and highlighting the benefits."

I've seen quite a few of the ads early in the journey when I was first using Duolingo and they're really good. They've really highlight well like, "Hey, you're doing great, but you can do even better." And they really highlight the product features.

And so I think that's a great opportunity for freemium developers who do have ads to be exploring that more to push that upgrade to the subscription.

Cem Kansu:

100%. And I think the other reason I like our ads is they're just better than programmatic ads. A user-

David Barnard:

A better user experience.

Cem Kansu:

It's just a better user experience. I mean, I think if I had a magic wand, I'd rather have no programmatic ads within my app and only internally produced ads because they look better. And we're also building lore for our world characters through, even though they might be talking about Super, they're also cracking jokes about their own.

For example, we have this controversial Super ad, I don't know if you've seen this one where Junior, one of our characters, is in a bathroom stall and he peaks and makes a roll joke because there's a toilet roll that he's peeking under.

It's weird, but it's like we're also entertaining it a little bit with the ads we produce because we have produced them with that intent. It's way better than showing programmatic ads.

David Barnard:

How does ad monetization do per user compared to subscription? And I would assume part of the idea with these partnerships and other things you're working on is to continue pushing that monetization better and better for those free users.

Cem Kansu:

So for us, if you look at Duolingo's statistics, subscription revenue is far and above the biggest chunk of our revenue. So, I think, it's above 80% and ads is a much smaller percentage. But if you look at payers about among our DAU base, actually MAU base, it's about 10%.

So 10% of our users subscribe. However, subscription is more than 80% of our total revenue. So we focus naturally when it comes to what we think about for monetization, mostly on the subscription side of things.

Because we believe it has one, it's already a much larger business, but also much higher potential because we still feel with any of these actually, but certainly still even with the subscription that has now become a sizable business, there's just so much more to do that is making the subscription better.

And also with the freemium app, you realize over time, and this is a lot of experimentation we've done, one is building the product to be better, so making our subscription features more valuable, but there's a lot of work to be done in also making sure you can communicate those well.

We touched on videos. Videos certainly help because that gives you a format where you can actually say, "Hey, this other thing that you could buy has all these features," because there's not a lot of other easy places to tell that story.

On top of that, there's how you design your purchase pages make a difference. For example, how users enter that flow makes a difference. So anyway, that's in my head how you're communicating your subscription.

So we're putting more and more of our monetization energy into those pieces, meaning make the subscription better and more useful and make sure we can communicate it well. That tends to have higher return for us in general as a business than ads. So most of our efforts is on subscriptions.

David Barnard:

Yeah. I mean, Facebook's numbers are just insane. I think I saw 250 per user per year in the US, which is about what a Duo Plus subscription cost, but the time and app, the way they're able to do their bid structures and things like that.

But I imagine there's still a lot of room to grow, but it's interesting that at Duolingo scale, the subscription opportunity is still so much better than the ad monetization.

Cem Kansu:

I think so. And I think with social media apps, the reality is it is fairly easier, I think, as a product problem to integrate ads into a feed.

And in a product where users are there, not for a particular utility, but they are there a lot of times to pass time or they don't even know why they're there. So I think it's way easier to increase ad load in a way that is not disruptive to how the user feels.

When you are there on an app for a particular utility and for us, a lot of users are there to learn a topic, well, not a lot, all of our users are there to learn something. I think increasing ad load is not particularly that tolerated, I guess.

So, I think, that creates a challenge and opportunity, honestly. For us, subscription just fits as a better business model than increasing ad load the way Facebook might be doing.

David Barnard:

Yeah. One last question on ads and then we'll move on to the subscription, but one of the product managers on your team came on the podcast I think 18 months ago and he talked about holdouts. That y'all do long-term holdouts.

Did y'all do a holdout on ads? Are there people today? Is there 0.001% of subscribers that still don't have ads that you can measure against the people who do have ads?

Cem Kansu:

Not anymore, but we did run, I think, almost a year or two year long holdout for ads. Wow. We do run long holdouts, although I don't recommend year long holdouts because what also eventually happens is you forget that you have a holdout and then eventually you introduce, since you're not testing it every day, you introduce a bug to it.

So even though you have a year long holdout, you realize at some point you actually polluted it because you like introduced bugs there. Anyway, so very long holdouts. We are anti just because of a practical perspective.

But we do way more like three month, six month holdouts on things that we know have different short, or we assume will have short to long term effects that might be different.

One of those, for example, this is not exactly monetization, but anything that has to do with our leader boards, we try to run long period experiments or long holdouts because on the Duolingo leaderboard set, if you start from the lowest league, it takes you 10 weeks to get to the highest one.

If we change the dynamics of how the leaderboard works, we might not see the actual result in two weeks. So we let these run for longer. Another one is push notification experiments where every, of course, new push notification you introduce has amazing returns because it has amazing novelty.

But then if you wait out six weeks, sometimes it doesn't just come back to neutral, it actually turns negative because you're sending too many push notifications that turns people off. So some stuff that we assume, not just monetization, might have longer term adverse effects, we run long holdouts or just run a long experiment.

David Barnard:

The way Duolingo test things is fast. We could talk hours just on that, but I did want to get back to the monetization journey. So at what point did it become clear that you needed to introduce a subscription?

And this was pretty early in the... Apple famously opened up subscriptions to all apps in 2016. I believe you launched a subscription pretty soon after that.

Cem Kansu:

'17.

David Barnard:

So what was the story from, "Okay, ads are starting to work, we haven't pissed off all the users, but now we're going to actually start charging."

Cem Kansu:

So the journey we went through was ads was the first experiment. And then after ads was introduced as a good PM, I was reading every review that was written on Duolingo that included the app store or Reddit or our own forums.

And there was a significant amount of people that were saying, "Can I please pay you guys? I don't want to see ads and I love Duolingo. I get so much value out of it." It wasn't a hateful, "I hate ads." It was more like, "I would love to give you guys a few dollars just to remove my ads."

And we thought that was a good idea and subscriptions were quite early for mobile apps at the time. You're right. Majority of the app wisdom was generally you want to sell IAPs. And I think games had obviously popularized that and that was pretty common.

Subscription apps were earlier. There was the big streaming and entertainment, but outside of that, there wasn't a lot of digital subscription behavior. So we took that idea to heart, but we said, "We're not sure if the subscription thing is for us." Our mind was still like, "We probably going to have an IAP model."

So we tested an IAP, which was honestly, looking back, it's so janky and it's not clear why people bought this, but we said, "You can buy an ad free month of Duolingo. So for 30 days, we'll turn off the ads, but it's a one-time purchase."

And we tested different price points, we tested $3, $5 and $10 and everyone thought this wouldn't work. Like I said, "Why would people pay this one-off bizarre exchange?" Honestly, the uptake was good and way more than what we expected.

So we were like, "Clearly there is some demand for turning off ads. Even though we created the most bizarre package to sell, people want to buy this, let's make it a subscription." But to make it a subscription, just no ads felt slim as a subscription service, so then we built offline capabilities.

So we said, "You can use Duolingo fully offline and turn off your ads." And then when we designed the purchase page, we're like, "This looks really weird with two features. We need a third feature." But we couldn't actually, at the time, figure out what the third feature should be.

So we wrote a line there that was the thing that users were telling us, which is support Duolingo in its mission of free education. I think we just said, "Support Duolingo's mission," for short. And that was our first entry into subscription, that was Duolingo Plus.

And honestly, what I think we saw happen was probably majority of users early on bought it for the fact that they enjoyed Duolingo so much and wanted to contribute back. The other half, or maybe the half of the demand was from no ads and minimally for offline. And that started us giving us a run rate.

Obviously, I mean, I'm happy to talk about it more, but from then to today, the subscription has evolved a lot, but that was the very first version.

David Barnard:

Yeah, that's fascinating. And I do want to go through that journey. So that is pretty light. I mean, again, this was 2017, so best practices and everything else hadn't fully evolved, and this was just a experiment.

But how did you start evolving that over time? And maybe just give us the highlights of the last almost decade now, eight years, the kind of biggest unlocks. When we added this to the paywall, when we added this new tier, what were some of those big unlocks over the past eight years that really moved the business forward?

Cem Kansu:

There's a few, and let me try to think, 10 years is a lot of time. There's a few levers that have, looking back, have really either caused tech function increases immediately or over time it really compounded.

So one of them, I guess now if you work in subscriptions, now this is a no-brainer, but at the time it really wasn't. Having free trials is one. I actually, as a PM, I resisted free trials for a long time. I assumed it would completely flop.

David Barnard:

Product was already free.

Cem Kansu:

Yes, exactly. It's like, why would people want to try free trials for the exact same product they've used for years? The product is free. If we were a fully premium product, free trial in my head made a lot of sense, but for freemium, like you said, I thought it didn't make sense.

Turns out I was wrong. Free trials are really helpful. For the obvious skeptic user that might not want to pay upfront, but they are interested in trying it out, it just makes it so easy to just jump in and try. It's very intuitive.

Now as a user, I love free trials because I'm like, "Well, I don't want to commit to pay. Let me give it a go."

David Barnard:

Were there specific things that were then behind the paywall that people needed to experience?

Cem Kansu:

Not really.

David Barnard:

Really?

Cem Kansu:

Offline, you can say offline, but offline is the same as online. It just works when you're in airplane mode. So if you imagine this subscription, which is no ads and offline, you didn't really need to experience it, but I think it just removes the payment hesitation.

So-

David Barnard:

Right.

Cem Kansu:

... it lowers the barrier to entry. So you're not committing payment. You're like, "I'll give it a go." And I think that's the real benefit of free trials.

So at this point, we were very pro free trials and I think our purchase flows are now designed to highlight how our free trial works as a mechanic. We even have designed pages which communicate, "Hey, this is a seven day free trial. On day five we're going to send you a notification to see if you still want your subscription."

And this has performed well in AB tests because I think users have a subscription aversion and I think in a lot of countries, maybe not as much the US, there's also subscription scam aversion where you click on something you somehow can't get out.

I think if you highlight what a free trial is, one, it's you don't have to pay anything really and out of what the mechanics of it, it relieves this subscription aversion.

David Barnard:

Yeah. Well, while we're on the topic, I know you recently did an experiment with a try for free button.

Cem Kansu:

We've been experimenting with a lot on everything our purchase pages, and maybe it's worth talking about. You mentioned AB testing. Maybe I'll take us on a tangent for 10 seconds.

So Duolingo concurrently runs somewhere around 400 AB tests. So the Duolingo app itself is heavily evolving real estate and a good chunk of those, not majority, but a good chunk of them also happen on our purchase flow.

So if you look at any element, whether that's Duo's flight path on the Super page or what the copy we write, how it's highlighted, how it's underlined, what the button color is, what the button copy is, all of those elements, we have teams trying to figure out how to make them better.

And of course, for the record, we're talk a lot about monetization. That's not the only place in the app where we're doing this. Our onboarding flow is the same. Our session end is the same. Our lessons are the same.

So we take maximal care into improving all these flows, but monetization is no exception. But that button copy, we've experimented a lot. And that is one learning that I honestly didn't have or didn't really expect. What you say in the purchase button makes a massive difference on your revenue curves.

We have tested many things. There's a couple magic strings that were, you look at the graph and you're like, "There must be a measurement mistake. Why are the numbers so green?" The two specifically were, we went from, I think, start my free trial.

So I think we said something like, "Subscribe." That's your baseline. And then we tried many things, but one of the big uplifts was, 'Start my free trial." That was very good. And then we tried, what did we say? "Try for free," that was better. And then we tried, "Try for 0.00."

Weirdly, that is the string that, I think, hacks your brain to really grock that button is entirely free. 0.0, two zero decimals. Now I think that's what's live in the app, if I'm not mistaken. And that performed really well.

David Barnard:

And just changing try for free to try for dollar sign 0.00, you saw a meaningful uplifting conversion?

Cem Kansu:

Correct. Yes.

David Barnard:

And it was, I guess, the start trial conversion, but then also the trial conversion to paid.

Cem Kansu:

Correct.

David Barnard:

So both numbers went up.

Cem Kansu:

Correct.

David Barnard:

Wow.

Cem Kansu:

Yeah.

David Barnard:

That's so crazy.

Cem Kansu:

Yeah. It was unexpected. Honestly, you wouldn't think this makes that much of a difference. I think one other thing I've learned with freemium, and honestly, any subscription and paywall behavior is a lot of...

When you go and design these flows, you are designing with logic, right? You are saying, let me communicate my features so whoever's going to buy should read them and understand what they're buying so they can do the logical exchange of, if I part with this many dollars, I will get this many features.

That is how you go and design a flow. But the reality is humans do not experience mobile flows that way at all, actually. It's way more based on emotion and reaction than, "Oh, let me read and think about the value exchange of what is happening here."

It's way more emotional. So I think these things that sound like they're saying the same thing, right? Zero and free are the same thing, but they create different emotions and make a difference.

David Barnard:

That's fascinating. Well, we've dived down multiple levels of rabbit holes, but to surface this back up. In thinking through the past 10 years, were there any other key unlocks, specific features that you put behind the paywall that really moved the needle?

Cem Kansu:

Definitely. So we talked about free trial. Another thing for us that was quite a meaningful turn in our evolution was honestly putting more useful features behind our subscription. So I started with saying, "Okay, we had ads and offline."

We started adding more... We added two features that were basically transformational to the subscription. One of them was Duolingo had this pacing mechanic called Hearts. Now it's called Energy, but Hearts was not integrated without subscription at all.

The way it works is you can go through Duolingo and do as much as you please, but if you make a certain number of mistakes, you start running out of hearts. And then before you can do new content, you have to regenerate hearts.

And you can do that by doing practice lessons or you can do that by watching ads. Or at the time we said you can renew your hearts with an in-app purchase.

We decided to integrate the hearts mechanic with our subscription and said, "Hey, if you subscribe, you actually get unlimited hearts." That was a big turning point because turns out a lot of users don't want to practice old content. They just want to do new content.

Even though practice is good for your learning, we said, "Okay, if you would like, you can choose your optionality if you subscribe." That was a good turning point because it made the feature set of Plus, Duolingo Plus at the time, way more integrated and way more useful.

Another feature we created was again around practice, custom practice. So we created a feature that is a separate tab called Practice Hub. We made that part of Super at the time. Actually, it's now became fully free. We now gave it all to free users because we realized it should be free. So we did that.

But at the time we said, "If you would like to do very custom practice on the things you've learned on Duolingo, here's an extra place where that's served to you as an inventory. So you can go back and practice anything you'd like in whatever order you like rather than the order we prescribe."

Anyway, this is a very long-winded way to say feature improvements that expanded the feature set of Duolingo Plus was naturally a win on two fronts, both conversion and retention of the subscription, which is great. We love that. And that gave us a lot of scale.

Other than that, honestly, it is rare to have put one silver bullet that improved things. For us, a lot of this has been continuous A/B testing improvement. I think we talked about the copy version as one thing, but imagine that happening across every bit and piece of our purchase flow.

And I think that's why Duolingo's purchase flows get copied a lot because I think people have heard us say this, I say this a lot, which is like we did A/B test many things.

So if you want to copy something, probably you're copying something that has been tested at least for Duolingo. It probably won't work for most apps, but you can look at us because we've tested stuff.

David Barnard:

That's fascinating. Before we move on, I did want to talk one more angle of monetization. I know by the time this podcast airs, you'll have taken something that was previously paid and moving it into the free tier. Why? Why do that?

Cem Kansu:

So let me talk about actually two features that we, one of them we just did and we are about to do. So Practice Hub I talked about. We have realized that number one, as a goal, we would like to provide more value to our free users because free users, there's the missionary reason why.

We want to teach more people, but there's the business reason, which is free users are our marketing engine. Duolingo grows as an entire user base and as a business through word of mouth. Word of mouth is much stronger if your free product is stronger because you might or might not pay, irrelevant.

If you're using Duolingo, you tell your friends about Duolingo and say, "This is awesome. I'm learning Spanish." They try it, they do the same. So that's how Duolingo really grows. That's the majority of our growth engine and that growth engine, of course, gets stronger if we can provide more value to free users.

So it's a delicate balance if you're a freemium product to decide how much value to put on each side, right? And we certainly want to put value as much as possible to our free user base. So that's the overarching reason on why.

But the two features that we did was Practice Hub and the second one is Explain My Answer. These are really, I mean, I love these features because they are really additive to your Duolingo experience. So Practice Hub we talked about, I'm not going to repeat it, that is now actually already live with free users.

And explain my answer is moving from being a dual-income Max feature, which is our highest tier to free users because we have realized the revenue value is not that high. So we want to... Meaning it doesn't make that much revenue.

David Barnard:

How did you determine that?

Cem Kansu:

We tested it. We tested giving it for free and compared it to a B condition or a control condition where it is part of Max. The impact, well, there was impact, but we decided to reduce our revenue, but rather provide value for our free users because we believe that's the right thing to do in the long term.

And there's one more reason, which is when we first introduced this feature, it was financially impossible for us to make it free because it uses LLM calls to explain basic... What it does for the record is you give an answer, you answered something, and we basically use LLMs to describe to you what you got wrong for your specific answer.

So it's very tailored. That's why there was an LLM that is used. LLM costs were really high so we had to put a part of a subscription just to provide the service, but costs have gone down significantly and we've improved how we cash things.

So since cost came down, our only decision was revenue loss and revenue loss we decided to take to provide this service for free.

David Barnard:

Yeah. It seems like that's a real theme is being very strategic about what's free and what's paid because of that balance, because you want to create that amazing free experience, but you still monetize. And so that tension must be difficult as a product team.

Cem Kansu:

Yes. And honestly, the hardest part is, for me, it's really easy. I think for our CEO, it's very easy. We always want to do the right thing for the user. It's not hard for us to think about, "Okay, how do we improve the free experience? There's some obvious things."

The hardest part is taking a loss on an experiment is not intuitive. So the teams that are working on these are finding it hard to be like, "I understand all this good stuff you said, but we're going to drop our metric." It doesn't feel intuitive.

So I think convincing the org and convincing the teams and explaining why this is the right thing to do long term is the hard part. Because you can imagine the positive side of what I'm saying doesn't really have a metric, but the cost of it has a very real metric.

So I think that they run hand-in-hand, but I think you have to realize, especially if you're a freemium business, is if you only always go by positive revenue metrics, you will burn yourself to the ground.

I don't want to call out specific companies, but you can see a lot of subscription businesses that have effectively nickel and dimed their users if they're freemium and then burned their growth engine, even though their revenue kept growing. But that only works for some amount of years, but eventually your growth stops.

I think for us, what's most important is our growth engine, our user growth engine keeps growing. That's why you have to make these hard calls sometimes to be like, "We're going to take a loss. It's okay because we're helping our growth engine," is the convincing part is the hard part.

David Barnard:

How do those calls get made when an A/B test shows a benefit? And I imagine this happens in more than just these monetization features, but how do you balance the art of product management against the science of product management when you run 400 simultaneous A/B tests?

Cem Kansu:

This is a great question. I think this happens a lot for us, I would say, because we have this other side of our equation as a business. So we talked about user growth and monetization that's the same for every app, but Duolingo has one more unique element, which is learning and learning efficacy.

So we run A/B tests to try to improve our learning efficacy. And the problem with that is it doesn't have a single metric you can put on a graph.

Even if you could, I mean, we use proxy metrics, there's no perfect metric for it, but even if you have proxy metrics that are going up, let's say your learning goes up, but your revenue comes down, should you exchange that? Should you not exchange that?

There's these trade-offs you have to make. And I think that's where you move away from mostly purely science to, like you said, the art of product management.

How do these decisions get made? If they are really consequential, it's like, all right, the revenue loss might be large or there's a DAU hit, whatever it is. So the clear business hit, we have to bring in most senior people to decide if as a company we should take this trade-off.

That means our CEO would be certainly involved in saying, "Okay, committed, let's do this." If it's at smaller scale, I think most of our teams understand these trade-offs fairly well.

Meaning if it's an exchange of, for example, clearly the thing we're doing helps you improve your conversation skills, but in turn, maybe it makes our content a little harder. So maybe it hurts some of our business metrics, but if the amount is small, teams can independently make those calls.

If the magnitude is large, but it's a strategic decision we should make, then I think we bring in senior people to sign off is how we operate.

David Barnard:

And this is why Duolingo is such a well-loved app. I mean, hearing you explain that and understanding how many product decisions have been made over the years to provide that better experience at the expense of DAU or monetization or other things, it's why people love it.

And that's like you said, it's hard to put a metric on that love and the social sharing and the feeling a user gets when they open the app, but it sounds like that's something you protect very carefully as a company which then pays dividends over the much longer term, even if you can't always A/B test your way to that.

Cem Kansu:

That's very well said. We have this operating principle that I love and we repeat a lot is take the long view. Meaning you can make business decisions very differently if your goal is to be a very profitable or a very fast growing company for two years versus if you want to build a company that will be around in a 100 years.

I think those are very different decisions. And to be specific, you could monetize the Duolingo feature set very heavily if you wanted to. And for two years, you'll make a lot of money, but one, that's not our goal and two, that would be terrible for free users, but three, we will burn ourselves to the ground if we do that.

We cannot build a 100 year business by taking more and more from... That's what a lot of freemium apps fall into this premium trap where the easiest win you can always do is like, "Oh yeah, let me take some free if you just make it paid." And immediately all your metrics are green.

And that does work for six months or maybe a year, but eventually it stops working. The other thing it does is it creates opportunity for someone else to do it for free. The Duolingo story, the very early one that we didn't really talk about is why Duolingo grew so much early on.

At the time there were tools teaching languages, there were the Rosetta Stones of the world, but they were paid and expensive. So being free and fun created a massive differentiator.

If we were to tomorrow wake up and say, "All right, we're going to charge for all of our great features." Very likely in two years, another great app will do it for free. So I think preserving the free value is just the moat that we have to keep and we want to keep for missionary reasons, but for business reasons it also makes a lot of sense.

David Barnard:

Yeah, that's such a great answer. Well, I did want to move on to the new products that Duolingo has been introducing and becoming a multi-product company. For a lot of apps, it's a big leap. The most famous example is Calm, Finding Sleep Stories and that secondary product market fit being this new lever of growth that really moved the needle for them.

So tell me about why you decided to start adding new courses and we can talk through how that went and what you added and everything.

Cem Kansu:

Definitely. Honestly, it's been a really fun journey and it's been now, it's been four years since we introduced our new subject, maybe three to four. But now we have math, music, and chess on top of languages.

The reason we chose those specific subjects, they're all varied and they have their own thesis, but at its core, they have a few properties that carry across.

One is they could be taught on mobile in a fun way. Not every topic qualifies. For example, coding, we considered, but it's really hard to teach on mobile, so we decided not to invest there. That doesn't mean we won't ever do it, but today it felt too hard to do.

Another one is they all have large markets of people that are already learning these subjects. So we're not trying to create the market necessarily, but we are, there's already existing demand. That's two.

And three, and I think this is connected to mobile, our expertise in language, which is at its core, if you think about it, a bite-sized, fun experience, can carry over fairly well to these subjects. The anti-example of this would be stuff that is done offline.

Running could have been something we decided to do, but that is an offline experience that you tie with an online experience. That is not exactly how we're used to doing bite-sized exercises. So we wanted to make sure our competencies played well in these subjects.

So these topics fit really well. We actually looked around quite a bit before we invested in each of them. Maybe the fourth internal reason these topics is each of them had one or more people inside the company that were quite tenured, tenured builders that really wanted to build these. So they became the seed for these projects.

David Barnard:

So I did want to get into the staging of this because you first launched math as a separate app, right? So what was the thesis behind doing that? And then what didn't work out that you now brought it back into the app?

Cem Kansu:

Yes. Our thesis changed actually over time. So when we launched math, we assumed our future would be separate apps. Our strategy, because at the time we also actually had, we still do, but we had another app called Duolingo ABC that is meant to teach literacy to younger children.

So teaching their first language rather than second language that Duolingo teaches. But our strategy was headed in a direction where all these new topics would be separate apps.

We thought at the time, this is partly true still, but our thinking certainly evolved that different audiences would seek different apps rather than be in one mega app for multiple topic learning.

So if you wanted to learn math, we thought the overlap between you and the language learner would be slim, and that's why you would probably want a separate app for doing one job. And I don't know if this was common with them at the time, but it felt really good as a strategy.

We did realize one thing as time went on, which is actually two realizations happened. One, it turns out a lot of people want to learn multiple things at once, at least Duolingo learners want to learn multiple things at once.

So it became an inconvenience actually to switch between apps. And for two apps, maybe you could survive, but if it's three, four apps, you're just not going to do it.

And there's just this inherent human demand that we realize exists within Duolingo, which isn't just like, "I'm here to learn Spanish." There's certainly a lot of users that are there for one thing, but also a lot of users are there for self-improvement in general.

So it was a natural behavior that we just saw happening, which is, "I'll learn a little bit of math, I'll learn a little bit of chess. I am there to spend my good screen time on Duolingo to improve myself.

"It is less relevant exactly what that topic is that might change over time, as long as I'm improving myself in something." That led us towards, "Well, let's make it easy for these folks to do that." So that was one.

The other one is just the business realities of growing a separate app is extremely expensive. While we were like, "Why are we playing hard mode when we can play easy mode? Because in the same app, you have a user base, it's way easier to get a user that is in the main-app to try something new than get them to download an entirely new app."

David Barnard:

It's so crazy to hear that from you. I mean, of all the companies with tens of millions of users and beloved and everything else, that the separate app was hard mode.

Cem Kansu:

It's hard mode, for sure. And I think a lot of companies have come to this conclusion. I think there's successful and unsuccessful examples of both of these actually, but the famous examples are, I think Uber and Facebook have done both separate and combined apps in multiple forms.

And that's why I don't think there's a right answer for every company, but for us, if you think about user behavior, which is generally, there's a lot of users that are around self-improvement and let me learn something and that will change over time, it just made sense for them to be in one app.

David Barnard:

Yeah. How did you solve the product problem? I think one of the challenges of mobile is that you have such tiny real estate to communicate everything, to shoehorn a feature into this tiny space that's already crowded by other things. How did you solve the product problem of it?

Cem Kansu:

With a lot of work is the short answer. And before we launched anything, these projects all took probably somewhere from six to 12 months to go from the first idea to full launch.

So that means in the meantime, we probably designed, tested, prototyped, redesigned, prototyped, tested again on ourselves and some beta users maybe many, many times until we got to the state where it's like, "Okay, within the confines of this mobile experience, this feels fun enough."

"It's actually teaching something useful and it's good enough to go in the app and be on par with the quality of where the language courses are." Because language courses have been improving for 13 years, whereas this new thing is starting from scratch.

So anyway, a lot of dog fooding and prototyping, I think is answer number one. But the other one is, I think, you can eventually teach a lot of things on mobile. There's some courses that are easy. There's some courses that are harder.

Music is really hard, I would say. Chess is fairly easier because you have one visual element that is the chessboard. As long as you fit that into the mobile experience, you can build a lot of stuff on it, but really it's now contained.

Music is way harder because we're, for example, teaching you piano skills on a screen and we have to think a lot more creatively on how to do that. And you can imagine the amount of stuff we've tried and still are trying to improve.

Which is you can't fit an entire keyboard on a piano, for example, and you also can't fit it when the phone is vertical, you have to turn it horizontal. To fit more key space, we actually tilted the keyboard a little a few degrees, so it gives you a bit more angle for touch space.

So all that stuff is innovation that you end up having to figure out. And I would say your music is the hardest. Chess and math are easier because math follows a very similar visual style to, in exercises, language.

David Barnard:

How do you surface it to users in the app in a way that doesn't get in the way of the people who just want to learn a language or are just there for music or crowding the screen real estate with these multiple things. How do you solve that problem?

Cem Kansu:

We've made it fairly integrated so it doesn't feel like you are crossing into a different app, but it feels like you are just switching what you're learning.

So we had already built a system basically to switch courses. So if you were learning Spanish, you could switch to learn French or Italian, any other language.

In that system, we now also offer with some different hierarchy, of course, unlike you could switch to French, which is the second language you learn the most, or there's your other topics now, which are math, music, and chess.

We integrated there and honestly, it's worked fairly well because we do already have a lot of users that learn multiple languages per day and they started discovering it. We also did some light promos just to raise awareness on the topics.

That said, this remains a bit of an unsolved problem, if I'm being honest, which is a lot of our users have no idea that we also teach math, music and chess.

So next year, we're still going to try to figure out how to make sure that at least our user base that's coming to the app every day is aware that these topics exist.

David Barnard:

Yeah. It's a tough problem. How do you think about competition with Duolingo? Do you think of YouTube and Netflix and other screen time as competition or who competes with Duolingo?

Cem Kansu:

I think any screen time is probably our realistic competition. If you think about the natural idea of competition, you would probably think, well, education, any education app.

But realistically, one, we are already way bigger than any education app. So at least user base wise. So we don't particularly see behavior of, "Oh, I stopped using Duolingo and now I'm using an education app." Most of any churn or any user that we lose goes to spend time on Instagram or YouTube or anything that is fun in their phone.

So for us, I think real competition is good screen time, or I guess, good or bad, I think that's subjective. But anything that is spent in screen time is Duolingo's competition. And that's inherently why we spend so much time on making Duolingo fun.

It is also, I would say it's headwinds to try to take something that is brain intensive. Learning a language is not an easy activity. It's not as passive as scrolling on your feed. It's actually burning your brain a little bit if you do it for a long time.

And we try to make it as easy as fun because otherwise, well, you'll go to Instagram, you'll go to TikTok. There's many engaging things that are just pulling you. So that's how... If I'm being honest, we don't really think about competition because it's so weird to think about screen time, but if you were to define it, it's really screen time.

David Barnard:

And I think that's honestly the case for most apps these days, unless it's a specific utility where you need to get in, get out, solve a very specific problem. If it's any kind of even habit forming app, I mean, almost any app, when you launch your home screen, are you pulled to Instagram or are you pulled to Duolingo to do your lessons? And it's tough.

How do you think of AI as competition and will AI just subsume everything? Nobody learns a language because they've got their AirPods automatically translating. I imagine that's a hot topic right now.

Cem Kansu:

Well, I get this question a lot and I actually really love it because I think we have a pretty clear thesis with two parts of this question. So one part is, will AI remove the need for learning languages? And in that case, will Duolingo... Well, what will Duolingo do if that happens?

The other one is, I think, will AI disrupt Duolingo because AI has language teaching capabilities? I'll start with the first one. I think the reality is being able to teach a language is not particularly a new part of a new ability for a tool to have.

What I mean by that is textbooks also can teach you a language. Now, AI can, which is for a technology perspective, it's incredibly impressive it can do that. But texts, it is not enough, unfortunately, to remove the need to learn languages.

People, we've learned... And I'm only talking about languages, we can talk about other subjects, that the reason people learn languages is incredibly varied. So you might be learning to connect with your heritage. AI doesn't really solve that at the end. You still will want to learn languages.

This is similar to when calculators came out. We didn't stop teaching math. It's quite similar. And the need to learn languages does not disappear even if you have massively good simultaneous translation and AI can do that really well.

So that's, I think, we don't see language learning demand going anywhere, and we certainly don't see it post LLMs either. The other question is, there are now other tools that are non-Duolingo, like the ChatGPTs of the world that can actually teach you language.

If you ask it to teach me Spanish, it will do something. And I think that's very fair, but I think that my textbook analogy still applies there where it can teach you a language, but is it exciting? Is it fun enough?

And until that is the case, a thing being able to teach you a language is not a threat to our business. I think we don't see that much and we continue investing in making things fun for that reason.

And maybe the third thing I will add is we see AI only as an accelerant to Duolingo because we use AI to do mainly two things that make Duolingo better. One, produce content at scale, at higher quality and higher volume.

The other one is introduce features that we couldn't do before that solves some of the teaching problems that we really care about solving like conversation practice.

So in a way, AI has helped us a lot and, I think, it will continue to, and we will continue building better products with AI, so we're happy in everything that's happening.

David Barnard:

Well, let's move on to a few of your contrarian takes. So don't build for a persona, build for the general population. Why? I mean, it's a classic PM thing. Build a persona, figure out who you're talking to, all that kind of stuff.

Cem Kansu:

Yeah, this is any PM framework would say don't build for a general audience, always pick who you're building for. I think when building consumer products, at least this is certainly Duolingo, I don't want to generalize too much, but we are very anti-persona.

So if someone comes in from the outside world, joins our team, most of the time their instinct is like, "Oh yeah, I'm going to build some persona maps. I'm going to call this person the Wander, the other one, the Explorer, and which one am I going to build for?"

And we're immediately like, "Please don't do that. Stop." Because I think if you want, it really depends on what kind of product you want to build. If you're trying to get to a billion users, which we have the ambition to, you have to eventually build for a general population. That's, I think, my theory number one.

Theory number two is with language learning, if you pick a persona, you're actually really cutting off the rest of your audience. And language learning is such a long tail of reasons for learning. Someone could be learning to get a job, someone to be learning to move countries, someone could be learning to connect with their heritage.

These are extremely varied. So if you pick one and say, "Oh, this is really important." Yeah, the common PM framework would say that's how you succeed. I think for us, that's how you fail because then you've ignored basically the entire long tail of why people are learning languages and you've built for a very specific one.

So if the use case of a product is varied, which a lot of consumer products have that property actually, you have to figure out how to build for the general population.

David Barnard:

That's great. Next one, people don't read.

Cem Kansu:

Okay. This is another favorite one. This connects to something I hinted at. Most early career, or honestly, even people who joined Duolingo with experience, product designers or PMs fall into this trap of what they write on the screen as copy and text, think is read by the user.

The reality is on mobile apps, people's attention span has already shrunk. You might be cooking with one hand and doing your app with the other hand. You might be on the toilet. There's all kinds of situations where now your attention span is shrunk.

And the other reason is you are going through an experience by not paying attention to every detail, but the core, maybe the 30% of a screen, so we always repeat this. Of course it's an exaggeration. Of course people read some things on mobile, but really in a graphical interface, they read very little.

But to simplify my extreme hot take is people don't read, so don't rely on text. Rely on great design to guide the user. And maybe a better said version of this is if you need to write a tutorial on what to do, you've already lost it. So it needs to be design that leads the user, not the text and the labels that you wrote.

David Barnard:

Yeah. All right. Well, let's close out with the three questions I ask every guest. What was the most impactful experiment, change or something you implemented this year, your biggest win?

Cem Kansu:

It's not exactly this year, it's been two years, but this is worth calling is the introduction of video call on Duolingo. I've been using Duolingo to improve my Spanish for nine years now. And this is the one feature I can confidently say it's helped me speak Spanish way more than before.

And I mean, we're very excited for what it can do. It still feels very early, but already if you use it, you can just speak Spanish better. And I feel great about that.

David Barnard:

That's such a great example of building a truly great product experience with AI. A lot of people are slapping it on and trying to put a chatbot in their app where it doesn't make sense. And the video call-

Cem Kansu:

I hate chatbots.

David Barnard:

... with Lily is such a great example of taking this new technology and actually creating a great customer experience. So it's cool that you brought that up as such a win. What about the biggest fail?

Cem Kansu:

Biggest fail is this is not... I don't think this went very public, but actually what a lot of our community doesn't remember or don't know is before video call, our first attempt at a big speaking feature was a human based tutoring product.

So it was a higher tier subscription because it was quite expensive for us to logistically provide, but you would press a button. I think we called it, did we call it Tutors? But it was our higher tier subscription.

But you would press that button, you would connect with a Spanish tutor on demand. No booking required. Honestly, it was pretty awesome. For speaking practice, it was really awesome. It had this one massive problem, which is no one wanted to do it. It was really scary.

Imagine this super fun experience and we would tell you, "Hey, you are going to connect with a human. You're going to practice Spanish." People would press it. They would dial in. We would see their faces come online and they'd be like, "Oh, shit, get me out of here."

Because one, it's really scary to speak in Spanish on the language that you're learning. And two, there's another real human on the other side that was extremely jarring. We iterated a lot to ease you into it, but eventually we gave up because humans just did not want to do that.

And actually, this brings me into another hot take that I do want to mention. Do not take user feedback at face value. For years, this is what people asked us to build. We're like, "Please build me a system where I can talk to a human tutor." We built it, no one used it.

David Barnard:

It's such a great example of a great feature, but just it doesn't work.

Cem Kansu:

Doesn't work.

David Barnard:

As product people, we can so get into our heads, take feedback and build something that's truly great, but it doesn't work. So that's such a great one. All right, last question. Growth would be easier if?

Cem Kansu:

I think growth would be easier if two things were true. One is pixel perfect design didn't matter in consumer because, I think, consumer app design bar has become so high that if you want to make progress and lead product led growth, you have to get pixel perfect design that will go out to users.

It can't be just a wire frame looking button that you put in your app or your illustration is not really cared for. This increases the time to ship products. So that's one. Pixel perfect design does matter and growth would be way easier. Maybe this was the case when apps first came out, but it matters and it makes a difference.

And I think two, if AI really contributed to productivity in both prototyping and coding. And what I mean by that is I think we are seeing today in the ranges of 10 to 20% productivity improvements. That's really impressive.

But when LLMs first came out, my hope was that they would be 10 X improvers. And I think, if you've got the Silicon Valley hype today, that's what the messaging is. But really I'm looking at what we're doing and what every company is doing.

Really the speed up of the product pipeline is in the 10 to 20% range. Growth would be easier if we got what was promised and it was 200% improvement, but I don't think that's the case.

David Barnard:

All right. Well, as we wrap up, anything else you wanted to share? Any jobs you wanted to shout out or anything?

Cem Kansu:

I will definitely do that. We are currently hiring for... Honestly, we're always hiring, but currently we're hiring for quite a bit of product manager positions.

I think the audience of this show probably is interested in monetization since we also talked about it and we're also hiring for monetization product managers. They're all on our creator's webpage.

David Barnard:

Nice. Yeah, I think that would be a great job for this audience. So if you're really into Sub Club, you're probably really into monetization, so apply. All right, Cem, it was so much fun talking to you today. Thank you so much for joining me.

Cem Kansu:

Thank you so much for having me. This is awesome.

David Barnard:

Thanks so much for listening. If you have a minute, please leave a review in your favorite podcast player. You can also stop by chat.subclub.com to join our private community.