On the podcast: The many failures of his recent app launch, the surprising results of his first-ever A/B test, and the many reasons why you shouldn’t plan a big app launch.
Top Takeaways:
🔄 Continuous evolution over big bangs: For subscription apps, frequent updates create enduring value, outpacing the impact of sparse, major launches. This steady stream of enhancements keeps your app relevant and signals relentless improvement to your audience.
🌱 Opt for flexible launches: Avoid putting all your hopes in one major launch. A strategy that includes multiple, smaller launches allows for adaptability and maintains your app's presence against the backdrop of an unpredictable news cycle.
📰 Press is unpredictable: Understand that media coverage does not guarantee app success. The broad reach may not always align with your target audience and many factors are outside of your control. Keep swinging, though, as some hits will indeed make a substantial impact — just keep your expectations in check.
💰 Adopt value-based pricing: Pricing should reflect what customers value in your app, not just the costs to provide it. Value-based pricing doesn't necessarily mean charging more, it just means charging the perceived value. Users don’t care about the costs of providing a service.
🔍 A/B testing insights depend on the nature of the cohort. The origin of your app's users — e.g. via launch events or organic growth — plays a crucial role in interpreting A/B test results. What did or didn’t work for one group isn’t necessarily applicable for the next — so test and draw conclusions appropriately.
About Guest:
👨💻 Growth advocate at RevenueCat and indie developer of apps like Launch Center Pro and Weather Up.
🍎 Although he’s neither a designer nor a developer, David has been building the kinds of thoughtful, intuitive apps he wants to use since the App Store first launched in 2008.
💡 “The tough thing about getting attention is you do have to do something unique… and that’s the trade-off. The calculus for me was, ‘Let’s wait and try to make a big splash with all these things.’ But really, we could have already launched the widget, and just adding interaction would have gotten attention.”
Episode Highlights:
[11:04] Just ship it: Don’t try to release a ton of new features at once — you’ll get more attention and benefits by releasing incremental updates.
[23:11] Failure to launch: What’s the worst that can happen on your app launch day? A major Apple announcement!
[32:09] Riding the wave: Offering a launch-day sale on your app is a great way to increase conversions when you release a major update.
[36:16] The value of value-based pricing: Set your app’s price based on your target customer’s perceived value of your solution, not your idea of how much it’s worth and costs to run.
[42:54] Dog-fooding the ’cat: David used several RevenueCat features (like Paywalls and Experiments) to set up and monitor the results of the Weather Up 3.0 launch.
David Barnard:
Welcome to the Sub Club Podcast, a show dedicated to the best practices for building and growing app businesses. We sit down with the entrepreneurs, investors, and builders behind the most successful apps in the world to learn from their successes and failures. Sub Club is brought to you by RevenueCat. Thousands of the world's best apps trust RevenueCat to power in-app purchases, manage customers, and grow revenue across iOS, Android, and the web. You can learn more at revenuecat.com. Let's get into the show.
Charlie Chapman:
Hello, I'm your guest host, Charlie Chapman, a developer advocate at RevenueCat, and with me today is RevenueCat CEO, Jacob Eiting. Our guest today is David Barnard. You know him as the host of this very fine podcast, but he's also a long time indie developer with his company, Contrast, and a growth advocate and my coworker at RevenueCat. On the podcast, we talk with David about the many failures of his recent app launch, the surprising results of his first ever A/B test, and the many reasons why you shouldn't plan a big app launch. All right. Hello, David. Welcome to your show.
David Barnard:
Longtime host, first time caller.
Charlie Chapman:
That's great. So, I'm Charlie. This is David, the longtime host of this podcast, first time guest. With us as well is Jacob Eiting, my boss and CEO of RevenueCat.
Jacob Eiting:
Yeah, I like the way you did it in the pre-show, which was and Jacob's here too. A good way to sum up my presence.
Charlie Chapman:
Yeah, the reason I am in the host seat and David's in the guest seat is because David just released a really big update to one of his apps and he has some learnings to use a corporate phrase here, I guess, to share some things that he learned as part of this launch process. We thought it would be interesting to walk through that. So, I'm here to try and fill David's shoes to walk us through that update.
Jacob Eiting:
I think we're liberated here, Charlie, to be a little more direct than we typically are with a guest that we've invited and on good terms to bring on here, but in this case, if he wants his show back, I think he's going to have-
Charlie Chapman:
This is a hostile takeover.
Jacob Eiting:
Well, I don't know. You are sitting in the host chair, so I think you have squatters rights.
David Barnard:
I'm an open book, looking forward to sharing whatever you want to ask.
Charlie Chapman:
Dangerous way to start this out. Before we get into Weather Up 3.0, which is the update that we want to talk about, I want to give everybody a primer of your background as app developer, not as growth advocate of RevenueCat. So, what is your story as an iOS app developer leading up to this Weather Up 3.0 launch?
David Barnard:
I was a struggling independent recording engineer in 2008 and not seeing my wife because I was working 2:00 PM to 2:00 AM and she was working 9:00 to 5:00. When rumors of the iPhone SDK started circulating, which I saw because I was an Uber Mac nerd working in a recording studio, I started thinking, "Hey, this could be a really good opportunity." So within days of Steve Jobs announcing the iPhone SDK, it wasn't even called iOS back then, I started a company.
I initially intended to code all the apps and started learning how to code and got three weeks into that and thought to ship something in the summer, I should probably find a professional. So, I hired a contractor and started building apps. I've launched 20 something apps on the App Store, seen close to 10 million downloads and millions of dollars in revenue. So, I've been at it a long, long time, 16 years now.
Charlie Chapman:
Starting from the beginning, you've gone through many different business model iterations of the App Store and I assume your apps have as well.
David Barnard:
Yeah, it's funny to me talking to developers these days who don't know the full history of the App Store and the fact that in-app purchase didn't even exist. So, early on, we did all sorts of crazy stuff, like a popular pattern for a long time was a free "light version" of the app, what freemium exists today. It's almost like your free trial and then you push people to the paid version of the app.
Charlie Chapman:
Which was always HD, or was that when the iPad came along?
David Barnard:
That was iPad. That was iPad.
Jacob Eiting:
Yeah. All right.
Charlie Chapman:
Yeah, I guess a little context for me too, just so you understand my weird references, I came into the iOS game a little later. I became an iOS developer and started building stuff in 2019 and I was actually Android until 2016. So, I learned most of my history through podcasts like this before I was an employee here. So, yeah, most of that is me getting that secondhand as well.
David Barnard:
Yeah. Then I did the whole free version with a ton of in-app purchases to try and increase the average revenue per user. The craziest experiment there, I launched a timer app and it was like 99 cents. It was doing fine and then I relaunched it as a free app with a ton of in-app purchases like, "Oh, you want this theme? It's 99 cents. You want that theme." In the app was an "ultimate bundle", which I think was 10 bucks, which at the time, it seemed totally ridiculous, 10 bucks for a timer app. But people bought the freaking ultimate bundle, because they just wanted everything. So, I was making 10 buck of-
Jacob Eiting:
It was a better deal, David. You'd be stupid enough to take the better deal.
David Barnard:
It was great. That was 2010, 2011, or something, just experimenting with business models and then started switching my apps to subscription in 2017 after Apple opened things up. I mean, that's how I ended up at RevenueCat and Sub Club Podcast and everything else was trying to switch my apps to subscription and it being such a huge pain in the ass, where we were spending way more time on the subscription app code and infrastructure than we were building a product. So, yeah.
Jacob Eiting:
Of those 20 apps, you remember when you first were like, "I'm going to do a weather app," but was it your first weather app?
David Barnard:
Technically, my third weather app. So, Perfect Weather was the first weather app I did, but it was like 2010, 2011. So, I've been building weather apps for over a decade now and then launched a whole brand new SKU, new app as Weather Atlas in 2017-ish and then Weather Atlas got rebranded as Weather Up.
Charlie Chapman:
Was Weather Atlas basically a rebrand then? You said it was a new SKU.
David Barnard:
No, no, no. It was a new SKU, everything. We wanted to switch to subscription. It was just such a brand new take. It's like file new project, take everything I learned from Perfect Weather. It was a new partner. That's another thing since I don't code, I've had this weird mix of working with contractors that I pay and then some combination of pay and do a rev share. I started doing the partnership thing back in 2010, 2011 after two projects went sideways.
So, each of those projects, I lost 50 grand. So, I was working with developers. They made all the promises in the world, but I was paying them hourly and the projects just drug on and on and on. I realized these aren't going to ship, stop throwing good money after bad. I just wrote off $100,000 that I'd paid out. That's when I started thinking, "Okay, I need to work with somebody who's got a little more skin in the game."
Jacob Eiting:
Developers without downside risk, there's nothing worse in the world.
David Barnard:
But then that has its own challenges. When somebody's working on it as a side project, they can get busy with their jobs and so it's been a struggle too working with that model.
Charlie Chapman:
So I don't want to spend too much time on the history, but I think one other thing that is probably related to the 3.0 launch for Weather Up is what was your marketing strategy for all of these apps then? You presumably didn't have a huge advertising budget or something at least initially. How has that evolved over time with all your apps?
David Barnard:
Yeah, it depends on the app. I've relied very heavily on organic channels. I've done very little paid advertising. Early in the App Store, I actually did spend thousands of dollars trying to make it work. With paid apps, it just was total money losing proposition. But mostly I've relied on getting attention, Apple's attention, press attention, and then ASO has been really big for my apps generally. Launching a pro was the top result for a bunch of launcher and other terms, so it got a lot of browse traffic. That's actually one of the things that we'll get to with a weather app, trying to get any organic traffic.
Those keywords are just so crowded, which makes it really hard. But one of my most successful apps was a mirror app and go figure at its peak, something like 10,000 or 20,000 people search the App Store for a mirror every day. I was capturing 6,000, 8,000 of those every single day.
Jacob Eiting:
As an ignorant person, what does a mirror app provide that the camera does not?
David Barnard:
So it's genuinely a good app and I made it embarrassing.
Jacob Eiting:
Well, I didn't imply it wasn't, David. You did.
David Barnard:
One of the main things is that the front facing camera, if you just launched the camera app, it's in a little window with all the chrome around it. So, this launches to a full screen mirror, you can zoom in. We did some really cool stuff. You tap a little flashlight button and it puts white around the edges of the screen and turns the brightness up to 100%. So, that in dark or low light situations, it's actually really helpful, one of those lighted mirrors and stuff, and then we had a really cool way where you tap the screen to freeze the frame and then you can save that as a photo. So, it's a cool way to preview your selfie before you save it. Little things like that, but people love that app.
Jacob Eiting:
I guess it becomes the thing you'd know to search for, right? It's interesting, the different niches will inform your strategy, how you can acquire users.
Charlie Chapman:
I think it's almost the opposite. Somebody's like, "I need to check to see if my lipstick looks right." They don't even think I can open the camera app. They're just like, "Is there a mirror thing?" and then you just search for it. You're trying to solve a problem and you go to the App Store. If the App Store is your default search engine, it can drive a surprising amount of-
Jacob Eiting:
Creating that loop. You know what I mean? Convincing people with normal human people problems to look for software. Let's just think about how magical that is for a moment.
Charlie Chapman:
Yeah, I'm always surprised when I look at my Appfigures, ASO tab and see that I'm ranking well and getting downloads from full sentences. I want a green noise. I make a white noise app, a green noise for free or something like that. People are just typing full-blown search queries. But yeah, I ask that because in the earlier era, getting attention if that's your big marketing strategy, I don't necessarily want to say it's easier, but it's definitely different, because Apple features in the early days.
Jacob Eiting:
No, it was way easier. How many apps are there now, David?
David Barnard:
Millions of apps now.
Jacob Eiting:
There's millions and millions. When I launched my first app, there were 270 apps on the App Store. You didn't need attention. There were 270 apps on the App Store.
Charlie Chapman:
Yeah, I guess people talk about literally scrolling the list of apps.
Jacob Eiting:
You could look at every single app. By 2011 or 12, I mean it was different, but still we were talking probably in order of magnitude, fewer apps than there are today. It's still to some degree getting attention, especially in the early teens, 2010, 2011, 2012, 2013 was the most predictable thing you could do. To some degree, it is and I think it's probably one of the better, more predictable ways to try to break through the noise if you know how to do it. But yeah, we should talk about it.
Charlie Chapman:
Yeah, especially if you have no marketing budget. So, yeah, let's talk about Weather Up then, and like you mentioned earlier, Weather Up was a rebrand of Weather Atlas you said. There's been multiple versions of it over time. So, when it came time to this Weather Up 3.0 launch, what was the state of Weather Up at that time?
David Barnard:
Well, I mean Weather Up 3.0 has been a long time in the making. As I mentioned, working with App Partner, my cousin, he was busy with other stuff and had a lot going on and so this has been a very long slow. It started in 2019 actually, like the designs for the widget.
Jacob Eiting:
That's when Charlie started doing apps.
Charlie Chapman:
Actually. Yeah. So, when I first opened Xcode for real.
David Barnard:
What kept happening was we would get deep enough into one specific feature and then just realize, "Oh, well, to get attention, we really should be doing X and we finished Y." I mean at this point, I actually regret not just shipping, so we were pretty far along with-
Jacob Eiting:
First lesson learned.
David Barnard:
... the Apple Watch app. Yeah, just ship. We should have shipped the Apple Watch app in 2020 probably when that was ready, but that was when widgets got really big. So, we're like, "Now we got to build a widget." So at WWC 2020, we're like, "Okay, we got to build a widget. We can't just launch the Apple Watch app." Then we're so close to launching the widget and then Apple announced interactive widgets. We can't just launch a widget when everybody else is launching interactive widgets. So, we should have just launched, but we held off. The tough thing about getting attention is that you do have to do something unique.
Well, widgets were really cool and unique in 2020 when they first were home screen widgets, but 2021, that's not enough to get attention. Apple Watch apps, complications, all these things and that's a trade-off. The calculus for me was, well, let's wait and try and make a big splash with all these things, but really at this point, we could have already had launched the widget and just adding interaction at the level we did would've gotten attention.
Jacob Eiting:
You'd be on Weather Up 7.0.
David Barnard:
Yeah, exactly.
Charlie Chapman:
So presumably then that long of an amount of time without major new features was impacting the business then or were downloads pretty much just swimming along?
David Barnard:
Yeah, I mean that's one of the big regrets. I was actually just looking at Appfigures recently and downloads were pretty consistently good and we had some nice spikes from getting attention or being part of big launches with new iOS features and stuff like that. All the way up until about 2019 when things went on hiatus and we were working towards this bigger update and things really dropped off the map. Apple stopped featuring the app because it hadn't been updated in a while. We stopped getting any bumps from these new updates and things like that.
So, coming into the Weather Up 3.0, I mean it's almost like launching a whole new app, because downloads were at low double digits a day, like 15 downloads a day. I mean I guess for some indie apps, that's a lot, but it's hard to build a business and make any money on that. So, the revenue was down. I've actually been losing money on the app for 18 months. I've been funding it out of my income.
Jacob Eiting:
Investing in the future, David.
David Barnard:
Investing in the future, yes. Again, I think that was one of the mistakes and that's the beauty of subscription apps too is that except for the getting attention, there's no incentive to hold back features like launching a big 3.0 where you're doing a paid update where everybody's paying you for the features. Well, nobody was going to pay me anymore because I launched those features, so I should have just launched them and kept moving.
Charlie Chapman:
Because even a really massive, big splashy launch amortized over the time between big launches, that time in between big launches is arguably a lot more important than how splashy the launch itself is. Unless you're just starting out, that's where it can make a really big difference to start from a big stepping stone.
Jacob Eiting:
I think the big launch philosophy has been one of the great mistakes of my career. There's so many incentives to be like, "Oh, yeah, we're just going to roll this together and do a big thing or we're going to put this all together in the fall." You're honestly justifying not shipping. You're being like, "I'm not lazy or I'm not scared. This is strategy."
Charlie Chapman:
For me, it's definitely fear, right?
David Barnard:
Yeah, exactly.
Charlie Chapman:
I'm not going to show this to the world because I need this one little extra thing, otherwise people are going to ignore me.
David Barnard:
I do think it's a legacy of box software really though, the mindset, and it did work prior to subscriptions when especially paid upfront apps, it works really well. Launching a pro would get massive spikes in revenue where it'd make like 60K in a week. Then we'd go along making hundreds of dollars a day, hundreds of dollars a day, and then we'd have another big launch and they would go up. But I mean, why is it even called Weather Up 3.0. What's the 3.0? Yeah.
Jacob Eiting:
Prior to that mindset because I think a bit too, it was all about building and packaging these big updates and then talking about leveraging that into love from Apple and the press. So, there were a bunch of these compounding things, but I think you're right that if you reamortize that over the time of value delivered, those features that you withheld for four months are not compounding or four years or whatever are not compounding over that period. I think it's really easy for that to overcome. Then of course there's always, I think, just the psychosocial aspect of it is that if you're not shipping, you're not shipping. Every day, you're not shipping your probability of never shipping goes up.
Charlie Chapman:
Especially in the Apple sphere, I think a lot of us have a tendency to think of our marketing like Apple's marketing and it's like Tim Cook can sneeze and that can be a news story. We are not that.
Jacob Eiting:
Do you have a Phil Schiller? You should get one then you could be Apple, but you don't have a Phil Schiller, right? Apple's marketing is world-class and they're professional and it's not just the brand and how it looks.
David Barnard:
That's built on decades and decades and decades of legacy.
Jacob Eiting:
They know the cadences, they know how to control the narrative.
David Barnard:
They are doing paid updates though. That's the thing is a lot of their launches are convincing you to get a new iPhone, and so it is like save it up, save it up, save it up, and then do a big launch.
Jacob Eiting:
They have hardware to do it.
David Barnard:
When it was box software, that was part of the deal. You had to hold back those features, and early in the App Store, you would release a new SKU as a paid update because there was no way to monetize that user base over time. I think a lot of what you all were just saying, I keep thinking yes, because subscription business model allows you to accrue that value over time and make it compound versus the old model of paid apps and paid updates didn't allow you to do that. Point taken., nobody cares about those things anymore and so don't hold off on shipping. That was a mistake I made was for the last four years getting trapped in that old mindset that did work in the past for me, but no longer works in the same way, because that's not how I run my business.
Charlie Chapman:
I think each small update is an opportunity to reach for attention and maybe not send an email to your press contact or whatever, because that you can oversaturate, but if you're constantly making small updates and each time you can make a little video or something of that update, every single one of those is a chance to catch somebody's news attention. Oftentimes what they're going to see is an old feature that just happens to be part of your demo. You're cashing in on an old feature or something each one of those times.
Most people aren't getting tired of you tweeting too often because most people don't see it in the first place. Then like you said earlier, one of you two is if you're constantly tweeting to the point of almost being annoying, it also signals... I get this with RevenueCat stuff. ... you guys are pushing so much stuff out. I can barely keep up, which shows that I'm on top of things. We're constantly coming out with new stuff. Even if you don't use it, that signal adds value, perceived value anyway to your product.
Jacob Eiting:
For a weather app like this, I mean I'm sure a lot of it's the mass market, but forgetting a start, forgetting anything written about it or whatever, you have to at least appeal to the connoisseur of apps and somebody's going to check how many updates there were.
Charlie Chapman:
So yeah, let's get to the actual launch itself then. So, what was your, I guess, strategy going into that launch? We already talked about the actual update. It was interactive widgets and the watch app, but what was your plan for getting attention?
David Barnard:
Press and hoping to get featured by Apple. We'll get into mistakes, but I did the same playbook I have for years was I know if I do something innovative, I can get some attention for that innovation. Then typically, when you do something really innovative with a feature like interactive widgets that Apple has been really pushing heavily and what's a big new part of the OS update, you can get attention. So, my goal for the launch was to get as much attention as I could from press, from Apple, from others, make some money, and then take that and start experimenting with Apple search ads. Then if that went well, start expanding marketing beyond that, but that's not how things went, which we'll get to.
Charlie Chapman:
Yeah. How are you trying to get that attention? I know most people listening know you have a lot of contacts. What was your general strategy to try and get those press articles and Apple features?
David Barnard:
People say, "Oh, well, you know people and that's how you get featured," but I don't think that's totally true. It helps. They will at least scan my email or at least see that I DMed them and maybe I have a tiny bit of baked in credibility because they know me or have known me for a while or whatever. But at the end of the day, and I think about this all the time, is that I'm not actually going to capture their attention unless I do something that they care about. So, for me thinking about reaching out to all my press contacts, I would be embarrassed to send them Weather Up 3.0.1 update with one tiny little feature that's not even interesting.
I still feel a ton of pressure to do something that's actually interesting in the market that they're going to be willing to write about, because I'm not going to push somebody to write about something that's not actually interesting. For this particular update, I worked on a video that demonstrated the features. Funny enough, Jake Moore, who we've had on the podcast a ton and done the tough stuff with the CEO of Super Wall, has been talking about for years how impactful videos are for paywalls.
With this interactive widget, it felt like it just something so visual that it needed a video. So, I was working on the video for the paywall primarily, and it turned out so good. That ended up being my press pitch. So, instead of hundreds of words, I mean you just have so little time to capture somebody's attention. So, the main way I pitched it was like a DM on Twitter. Hey, I'm working on some really cool interactivity for widgets, some weather. I think this is really interesting. I got quite a few people who were like, "Yeah, I don't really cover apps anymore," or-
Jacob Eiting:
How did you get this number?
David Barnard:
I wouldn't cover an app, but they saw the video and were like, "Wow, that looks really cool." So I think whether or not you have a relationship with somebody, because a lot of the people I pitched, I didn't know and they didn't know me, but it's all about communicating that value and it being something that's actually worth them writing about. That video really helped.
Charlie Chapman:
Yeah, because it's not just value to the user, it's also interest for a story, which are slightly different things.
Jacob Eiting:
You got to help them get their story written for the week. They have a job, they have a quota, they have a number thing you have to produce, and if you want them, help them do their job and find something mutually beneficial for all three parties, the reader, the journalist, and you. If I was to say any distillation of how to do PR, it's that. Build something to win to all parties and it's hard. You have to find something they're interested in. You read about what they do, you learn about whatever.
Well, I mean this is February of 2024, which every tech and otherwise online outlet in journalism is disappearing right now on the daily. So, maybe you're the last one to get this in before writing about apps just doesn't exist anymore. If you needed any more signs that this model of launching is no longer, that might be it.
Charlie Chapman:
To that same point, it's not just is it interesting, but is it interesting currently? You had a little bit of an uphill battle. Well, you had a huge uphill battle, which we'll get into a second, but even ignoring what happened on your launch day, you weren't launching this when iOS 17 was coming out with these interactive widgets where-
David Barnard:
Huge regret.
Charlie Chapman:
... people in the press were literally begging, please send me your interactive widgets.
Jacob Eiting:
Yeah, I need something-
Charlie Chapman:
Yeah, they needed it.
Jacob Eiting:
... to ship about and I don't want to just copy Apple's press release.
Charlie Chapman:
Right? You had to pitch them on. Was this January when you were starting to reach out?
David Barnard:
Yeah.
Charlie Chapman:
Yeah. So, not only did you need it to be compelling in its own right as a story that they already want to write, but you had to make them want to write it well after the readers are demanding this. Anyway, let's move to the actual launch, because this is where things get pretty interesting. So, I guess I'll just let you take it away, but how was your launch day itself?
David Barnard:
Terrible. Really, really bad.
Jacob Eiting:
At least you didn't wait four years for it, so that's good.
David Barnard:
Yeah, I know after four years of hard work of all the things to happen, I had scheduled the launch, put it on my press page. So, if you go to press.contrast.co, we'll link it in the show notes, but I create a press page for every release that I'm trying to get a little press for, and on there, I put the launch date and time. So, it's the embargo time. I did talk to a few journalists and I had set this time and it turned out that at the exact same moment Apple was planning on announcing their DMA compliance. Somebody from the press reached out and said, "Hey, there's big news coming an hour from now. Can I just publish now?" I was like, "Oh, shoot."
Charlie Chapman:
Yeah, that's not a good sign.
David Barnard:
It's like, "All right, just go for it." Then I reached out to other folks that I thought might be writing about it and told them, "Go ahead and write about it." Then even 9to5Mac was like, "Hey, I think we should just wait until tomorrow. They didn't even publish till the next day." Then several other people I was talking to got so distracted with the DMA news that they didn't even write about it. So, there was layers there of one, it didn't get written about by as many outlets as would've likely written about it had it not.
Then two, there wasn't any social halo, because the air got sucked out of the room. So, people who may have seen it and may have otherwise tweeted about it or said, "Hey, this is cool," or other journalists see TechCrunch write about it and they write their own story. Instead, everybody was super busy.
Jacob Eiting:
I didn't even watch that day, David, to be honest. I was so busy. I think on a normal day-
Charlie Chapman:
I was going to say, even including you, David. You are one of the people that people go to Twitter or whatever to find. So, it's like they go to your Twitter and they might see like, "Hey, I launched this new thing." They're just scrolling to find your take on this new thing.
Jacob Eiting:
This is why I don't discourage side apps explicitly because the world just discourages side hustles. Your main gig will always destroy anything you try to do.
David Barnard:
Yeah. So, an hour into my app launch and I didn't know what the news would be, but an hour into the launch, before I even had a chance to tweet it from the company account, before even some of my own launch day plans were finished, Apple announced that and then I was all hands on deck with you and Jacob and everybody else at RevenueCat trying to figure it out and then scheduling a webinar for the very next day and reading through all the documents and chatting about it internally. So, what would've maybe otherwise been a lighter RevenueCat day where I take a little personal time to celebrate my app launch was just complete decimated.
So, instead I was up till 2:00 AM reading the fine print of European regulation to host a webinar the very next day. It was just about as big of a disaster as you could possibly have when your plan is to get attention. Apple still hasn't featured and I'm not totally sure why, but I have a feeling it's that... Well, two things, and this is a regret that I have now looking back, is I was trying to squeeze it in ahead of Vision Pro. I thought so much Air was going to get sucked out of the room from Vision Pro. What I didn't know was Apple was distracted with the DMA, but the whole editorial team was I'm sure so busy with the launch of the Vision Pro.
I never could have known that the DMA would be such a big deal and would be launched on that day, but I should have known that the App Store editorial team would be super busy with the Vision Pro. The reason I was trying to get it out was to launch, was to actually ship versus like, "Oh, now we need to build a Vision Pro app before we can ship." So I was trying to get out before the Vision Pro so that people weren't asking, "Well, where's your Vision Pro app or whatever?" Well, it turns out a month later, I don't think anybody would ask for a Vision Pro app anyway. So, hindsight 20/20, I should have just waited a month and let the Vision Pro stuff blow over before launching.
Jacob Eiting:
That's a top tip. Sub Club listeners, if you're going to launch your app, don't do it on the day that Apple announces the DMA. So, take that one, write that one down in your notebook.
Charlie Chapman:
This is really actionable. This applies to all businesses universally.
Jacob Eiting:
You'll do better if you pick a different day than the day that the App Store melts down in Europe, but it's another side effect of the big launch. If you put all your eggs into one big launch basket, sometimes you're just going to catch a stray. Some big news article is going to come out. It happens with fundraisings too. Companies put big efforts behind their fundraising announcement and then that morning whatever company files for bankruptcy and the entire startup news scape is covered with that. Nobody cares or some other company raises more or whatever. That's just a risk. There's sometimes a little bit you can do to de-risk that.
Maybe you have a friendly contact at the press that tells you, "Hey, there's already an embargoed news drop that day," but you're not usually that lucky. But if you're launching more often, you're derisking your launch plans by having many small launches. You can always try to do both. Why can't you just have many small launches and big launch too, right? I think you can do.
David Barnard:
Yeah, that's what I should have done. I could have done that. We could have launched the Apple Watch app and it would've been a decent launch.
Jacob Eiting:
Just launch it quietly, blah, blah, blah, blah, and then be like, "This is Weather Up 3.0." They've never seen it. These journalists have never seen it. You can lie. I don't know.
David Barnard:
That's what I was going to stay. You're saying the big launch, and I think big launch maybe has two meanings. There's the big launch in the sense of you're holding a bunch of stuff for this one big launch, but there's also the meaning of making your launch big, going for just being loud and everything. You can be loud once a month.
Jacob Eiting:
I mean, this is the wise advice. If you have a launch that goes poorly, just launch again. The definition of a bad launch is nobody sees it, so nobody's going to see you do it again.
David Barnard:
RevenueCat do that?
Charlie Chapman:
I think you said that on stage somewhere. So, I think that's okay.
Jacob Eiting:
Yeah, we posted on Reddit and it got some steam, and then we launched a beta. Every time we fundraise, we're like, "RevenueCat's launching out of blah, blah, blah." We make some stuff up and it's a little bit of a wink and a knot. We're always doing stuff. I guess it's a little bit of showmanship just to put a narrative to the building you've been doing. I mean, look at Tesla. Every big company juices up stuff they've been working. They dress it up, they push it.
David Barnard:
Every Apple keynote starts with a bunch of things that sound like launches that only all the super nerds are like, "We've heard this before," but most people watching it are like, "This is new. This is okay. Okay, here's a new laptop."
Jacob Eiting:
[inaudible 00:30:16] who are watching, which is probably who it's for. But yeah, I mean it's the power of narrative control. You can really drive the narrative, and I was telling you guys before this started podcast, this is all made up. All it's made up. Contrast is made up, but nothing you can do to control for just dumb luck.
Charlie Chapman:
It feels like in this particular case, the mistake, if you were to say that, would be you made a lot of ride on this single event and then things that are outside of your control can go wrong. You gave up on all of the smaller events that could have happened over the last three years or whatever and put it all into all your quivers behind or I don't know the phrase, something with a bow and arrow and putting everything behind the one bow or arrow.
David Barnard:
All my eggs in one basket.
Charlie Chapman:
Yes. So, that maybe is the mistake that happened here, but I guess maybe the more interesting thing then is what now? What are you going to do? Like Jacob said, are you going to try and relaunch?
David Barnard:
Short answer, yes. I want to actually go through more of the failures of the launch. To give the audience a little bit of context, Jacob and I were chatting Friday. We were talking about the podcast broadly and we've had some really great guests on, but one of the things I want to do more is talk to people who have failed and talk about even the ones who have succeeded about all the failures leading up to the success. I was like, "Yeah, I've failed so bad with Weather Up." Jake was like, "Well, let's talk about it. If we're going to get other people on the podcast to talk about failures, why don't you lead by example?"
So this is Monday three days later and that's exactly what we're doing. So, I actually want to talk through more of the things I think really failed at the launch before we get to how I hope to potentially recover from this failed launch, because I think it's really instructive. Then for me doing all this stuff, it informs the questions I ask on the podcast. I'm using RevenueCat experiments in the app. It informs how I talk to my colleagues about our product internally. I give tons of advice on Sub Club. I get tons of advice on Sub Club.
Jacob Eiting:
Apparently, he doesn't listen to all the advice in Sub Club.
David Barnard:
I didn't take all the advice. I think one of the other big failures I wanted to discuss was pricing and specifically because we talked so much about pricing. I just failed at every level of the typical advice that we give. So, one of them is pricing is not as important as you might think, and it's probably better to launch with a lower price and just get more people using the app and get more subscribers. I launched with what is probably the highest ticket weather app subscription at $4 a month was fine, but the $40 a year. But interestingly, my biggest regret is not the $40 a year. It's that I didn't do a sale on launch.
So, this is the advice I would have, if you are going to do a big attention grabbing thing, do it with some discount, 25% off, 50% off, whatever, because all that attention I did get, most of the people who wrote about it listed the price. A lot of people who maybe even read the Tech Crunch article, read it on 9to5Mac, read it on MacRumors or whatever, Boy Genius Report, they all saw $40 and were like, "Eh, no way." But if they had seen $40 but 50% off, I think we would've done a lot better.
Charlie Chapman:
Especially time, it creates that urgency like take action now. Don't just wait.
David Barnard:
Part of the reason I wanted to push the envelope is that I talked to so many folks here on this podcast and we talked to so many people inside RevenueCat where they have what seemed like unreasonably high prices. My theory now is that, well, one, this is a weather app, it's a commodity good. I should have really better understood willingness to pay. I should have looked at what other apps are charging and everything, but the other thing is the source of attention matters a lot. So, the apps that are charging a higher rate, they're either creating a lot of demand through paid advertising where there's not a lot of expectation going into it versus getting press.
The attention you get I think makes a difference to that willingness to pay and the type of people you attract. Then especially apps that are really big with search, if somebody's searching for a weather app and is looking for a solution, that's a really different audience than people just seeing it in the press and wanting to check it out. So, there's a different intent, a different user, all those things. So, I think it was a bad move pricing it at 40 bucks. I think we probably should have done $30 and then done a discount to $19.99 for the launch.
Charlie Chapman:
Yeah, it's interesting. Almost definitely this thought is coming from me listening to this podcast, but the idea that the journey that the user's taken at the point when they see the paywall being super important, which oftentimes is when you're talking about the onboarding experience, but it's also the attribution as well. It's interesting to think that whether that person came through a splashy launch or press versus the high intent, they were searching for it or they saw an ad on Facebook or whatever, that should also drive what offering you're showing them on that paywall.
I don't know if I've ever seen this before, but it would be interesting to... Oh, I guess I have. On a launch itself, all the press people that you are trying to get to write an article include a link that has the attribution, so that discount only applies if they get it through their article.
Jacob Eiting:
If you're launching an otherwise unknown app with a day of press, you can pretty much guarantee anybody who buys it on that day is coming from the press. You could do a time limited sale.
Charlie Chapman:
Even if 100% of users, that's the case, from the person writing the article story, being able to say-
Jacob Eiting:
Just tell them.
Charlie Chapman:
... 9to5 readers are the only... Well, yeah, that's true. You could just say that.
Jacob Eiting:
Am I coming off unscrupulous? Nobody's going to mind. It's literally true. If they click, they'll get the discount. Maybe everybody else gets the discount too.
Charlie Chapman:
I think that's something that could live on though, right? One of the things for me that I do see in the data, and also it's in my head that this is a thing, is a lot of these journalists for some of the bigger press junkets, they spend a lot of time on SEO. They make these big listicles, because they know that one, they'll get a lot of attention on that day, but then they'll get attention for the next six months when somebody buys their new iPhone and they're like, "All right, what are these new widgets?"
Because that's probably when they actually update their phone at the same time. That's a driver of traffic over time for your app as well. Each one of these articles that they write, that deal can live there as well. That's also part of that narrative. That's user journey all the way to your paywall. So, theoretically, those could even live on there and show a different offering to somebody that came from an article versus somebody who came from searching I need to know the weather or whatever in the App Store.
David Barnard:
One other thing I did want to mention on pricing is that I priced based on things that I knew and based on my thinking of the value of Weather Up and my costs in servicing Weather Up, not on the customer perceived value of Weather Up. I think that's a huge thing that founders can get themselves tied up in knots about, and I succumbed to it myself, is that one of the features that we were leading with that I don't think many people are even using, people didn't even talk about it much but I was personally so excited about it because this is how I use it, is that with Weatherup 3.0, you can set a different weather source per widget.
So, I have a whole home screen dedicated on my iPhone with three widgets. It's Apple Weather as a source and AerisWeather as a source and AccuWeather as a source. So, at a glance, I can see whether any of the forecasts are showing rain coming. I can see the differences in the forecast and everything like that. Super uber nerdy feature. Well, it's also very expensive to do that, because every 15 minutes or whatever, it's hitting the server not one time to one weather data provider. The weather data is why I've had been losing money on the app leading up to this launch was that it's just really expensive to provide weather data and the widgets and complications are especially expensive because they're constantly updating in the background.
So, I got myself all twisted and knots thinking, "Okay, we need to make sure there's a level of profitability here. If a bunch of people start putting five widgets on their home screen with different locations at different sources and everything else, it's going to get really expensive for us to service these customers." It's really hard to educate customers on how expensive weather data is, and then the fact that it's even more expensive because-
Jacob Eiting:
Just give me my clouds, weather man.
David Barnard:
Nobody cares, exactly. Nobody cares.
Charlie Chapman:
Don't tell me about your business model. I don't want to know.
David Barnard:
So I was pricing based on my knowledge of that and my own thinking the feature was going to get super hyped.
Charlie Chapman:
You only have the one tier, right? It's not like you have a more expensive plan.
David Barnard:
I didn't have multiple tiers. Exactly.
Charlie Chapman:
So you had to model it in some percentage of users are going to be doing all of this.
David Barnard:
Exactly.
Jacob Eiting:
I don't think we talked about, but you didn't put any product analytics, two causes of that there, but one, you didn't have product analytics to tell you how many users are doing this, which you could probably look at the API calls to see the relative now. The percentages, which then I'm going to bust you for, that's another symptom of not launching. If you had launched this as a standalone thing, you could have not touched anything, just measured how many people are doing what, and then you would have some idea of like, "Okay, actually as long as we charge X, we're going to be able to cover costs. Not that many people actually use this feature." There's some risk there.
David Barnard:
This is why we talk about all this stuff. I should have surveyed some users.
Jacob Eiting:
Value-based pricing is typically helpful. I think actually in this case, David, value-based pricing is hurtful for your bottom line potentially. Actually not, because I think if you're closer to the perceived value of the product, you'll get a better optimum. Value-based pricing, you want to set your pricing to the perceived value for some aggregate of users as opposed to cost-based pricing where you're saying, "It costs me X dollars to deliver. I'm going to charge x plus 5% profit or whatever." In SaaS and stuff, it's really important because the cost to deliver is often very, very low. If you price just to your cost, you're giving away too much.
Often, especially in SaaS where you're as a developer having to ask for $100,000 a year, you really have to be like, "Gosh, that's so much money for software." But then you go like, "Well, if I'm replacing a developer or if I'm replacing some amount of headcount or some amount of whatever, now I see the value." The value is there and therefore you can feel it. But you could be on either side of that price curve in the sense that you're pricing way above the perceived value, the demand's going to drop off again. So, value-based pricing doesn't necessarily mean charge more. It just means charge the perceived value, because the costs are only something really the provider of the service cares about.
Charlie Chapman:
Well, and they're highly variable based on the number of users, which is weird too.
Jacob Eiting:
Also, at the stage, there's some fundamental costs that you can never get below potentially. I mean, when RevenueCat launched, our cost structures were really brutal in the early days, but it's okay because eventually we're going to have a team that's going to focus on making everything efficient and driving down costs or whatever. But in the early days, it was just like, "Turn up the database dial, just keep burning money." But obviously, a different thing where APIs, you have a more fixed cost, but I think the worst way to do this is to be like, "How much did it cost me to build this app?" Then they divide into that and it's like, "No, don't do that. Don't do that. That's not going to end well for you."
David Barnard:
Then the bigger problem for me too was on the value-based pricing front is that people have expectations coming into a weather app. Understanding my market as weather nerds, who's going to put this widget on their home screen? Weather nerds. Weather nerds already have five other weather apps. They're probably subscribed to two other weather apps. So, they have a sense of what a weather app should cost. One of the ones I got compared to, which I know Brian, the developer of Carrot Weather, and that's one I got called out against, was like, "Well, Carrot Weather has more features and is only $30 a year on their most premium tier." It's true. If I would've thought more about who I was marketing to, I would've realized that they already have that price comparison in mind.
Jacob Eiting:
Yeah, it's a real thing to assume the market is already somewhat in equilibrium, especially for something where you have a lot of products. If you're building something truly new, it's harder. But if you're building something that's like, I don't want to say a commodity, but I think car companies are super precise in their pricing, because most cars are commodities. They're slightly differentiated. So, they have to be laser tight with where the market is, and they know they won't sell them if they cost more than this and stuff like that.
David Barnard:
You got a price in that differentiation too, right? I think that was a little bit of where I was hoping that this was differentiated enough and valuable enough. We got thousands of subscribers at this launch. It wasn't a complete failure. So, I mean, I did prove that a certain number of people were willing to pay a premium for what they perceive to be a premium feature and a premium experience. Now, I have the opportunity experiment.
You'd asked earlier about, "Where do I go from here?" we already covered it. I'm going to launch again. Am I going to get as much press this time? Probably no. But like you said, I'm going to create one or two features, nothing massive, but one or two features that I think are interesting enough to get some more attention and bundle those up and do a little relaunch. Some people will be seeing it for the first time-
Jacob Eiting:
Yes, with a sale.
David Barnard:
I'm going to put it on sale. Exactly. Exactly. Then from there, then I can experiment more with pricing and win back offers and things like that.
Jacob Eiting:
So you ran on a shill here for a minute, because you used a bunch of RevenueCat stuff. You ran into experiments-
David Barnard:
I did.
Jacob Eiting:
... and things like, "Well, how did that stuff go?" That's probably more than most folks do on a launch, but what did you set up? What did you use and how did that go?
David Barnard:
Yeah, I mean, I guess as a guest, I get to show revenue.
Jacob Eiting:
I want an honest customer testimonial.
David Barnard:
Paywalls went way better than I expected. So, we use the RevenueCat footer view where we built a custom SwiftUI paywall behind where we're able to do the video, do a lot of custom fancy stuff, which probably wouldn't have made that much more money. We'd just used one of the RevenueCat templates, but I wanted to do the video on the paywall and everything else. But it was really nice being able to let RevenueCat just do all the logic behind purchases and everything else. So, it saved us a ton of time. I think Brock, who did the development, I think he said the super fancy SwiftUI paywall took less than a day and then all the logic and everything else-
Charlie Chapman:
For context, if people don't know, we have a paywalls feature at RevenueCat where we can render everything for you and it's all driven by the backend. You can change things, but there's a version of it where the only UI we render is the actual hard part, which is the prices and the CTA at the bottom and that stuff. Then you can build your own super customized, fancy, animated marketing view behind it. So, that's what you used there.
Jacob Eiting:
It's a great solution for the developer that wants to still own the paywall, but you also want to save a bunch of time.
Charlie Chapman:
Yeah. Well, and selfishly from our perspective, it also means that we do support any bespoke feature that we haven't built yet ourselves into our feature. So, at the beginning, we didn't have videos, or if you have certain types of slideshows, even if it's not super complicated, if we haven't built it yet, it is possible.
David Barnard:
I don't think he had to do anything, Brock, code wise, because once we were using paywalls to power it, I was able to kick off an experiment. So, at launch, I scheduled it, which that's a new feature, scheduling experiments. But I scheduled the experiment to start midnight the day of launch or 12:01 AM the day of launch. The experiment I launched with, it was so fascinating because RevenueCat shipped experiments 18 months, two years ago. I've talked to people who've launched experiments. Charlie, you wrote a blog post about using experiments.
A lot of people have talked about experiments, but launching an experiment that I conceived of, implementing it myself, it was just such a different experience of dogfooding RevenueCat. Then also, again, podcasts and the advice I give and what experiments to run and all those kinds of things, it was so fascinating to set up an experiment myself, run it, look at the results, analyze the results.
Jacob Eiting:
Real skin in the game too, right? It's easy to talk about somebody else's decisions and tests and be like, "Oh, you should do blah, blah, blah." That's what I do all day. But yeah, when it's your money and you're like, "I'm actually testing this," it's different. So, it just worked though. That's pretty cool, which it should. I would say unless you're putting something embedded in the behind the footer view and the custom view, the footer will just automatically adapt to the offering. Even just customizing a paywall for having two different offerings without using our stuff, which it's not insurmountable, but it's days of drudge.
David Barnard:
This is the first A/B test I've ever run in 16 years of building apps, because it was always such a hassle. I didn't want to pay for SaaS to run A/B tests. I didn't want the developer to spend months building out some bespoke system. So, the only way I've tested in the past, which we talk about this on the podcast and I think is a valid way to test, is you launch and you look at the numbers. Then a week later, you update the app with something very different, and then you look at the numbers that week. That's a valid way to do it, but this is the first time I've ever done a real A/B test because it was so easy to do. So, the test I did was defaulting to annual being the pre-selected option on the paywall versus monthly being the pre-selected option on the paywall.
Jacob Eiting:
Oh, it was that subtle.
David Barnard:
It was that subtle.
Jacob Eiting:
So it wasn't even different price. You were just like UX different, which you can test with RevenueCat paywall.
David Barnard:
So just what's the default? Part of the reason I did that is that I was considering just making the monthly the default always, because we talked to folks here on the podcast about how monthly shortens that feedback loop of people who are staying engaged with the app and churning quickly and everything like that. Then I also wanted to get some monthly recurring revenue built up because we have monthly recurring costs. So, do we do a big launch where we get a bunch of annual and then we need to budget that over time to make sure that we can the weather API bill paid, or do we try and get more monthly revenue where it's more consistent over time?
So the results were fascinating and I'm looking at the results right now, but the headline is that realized LTV less than a month in, so I don't have renewals yet on the first renewal month, but realized LTV is 60% lower on the people who saw monthly. That makes sense because a lot of the people who saw monthly just stuck with monthly. But I was running some numbers with my colleague, Dan, and this is something that would be really fun for us to model at some point, but assuming a normal level of churn, I'm unlikely for the realized LTV to cross before the year mark. Then at the year mark, those annual renewals will kick in and the realized LTV will shoot back up again. So, basically, I just made less money by launching with this.
Jacob Eiting:
Well, no, you learned about something that definitely doesn't work, right? You mapped out the space.
Charlie Chapman:
Exactly. That probably doesn't work. I think the interesting thing here is when you stop this experiment, you'll still be able to track these cohorts for the next, I think, 400 days or something. I would go back and look at these results after that year mark when your annuals have actually started renewing and then look at your actual lifetime value, not over 30 days, but over that full period.
David Barnard:
Yeah, 366 day of lifetime value instead of just the first one.
Charlie Chapman:
Yeah, because that's something that you can make a lot of inferences based on other people you know and what their annual churn rate looks like versus monthly. But at the end of the day, you don't really know until you see what yours looks like.
David Barnard:
Yeah. The monthly renewals will start coming in this Thursday actually. So, I'll be able to see pretty quickly, but in revenuecat.com, you can go into the subscriber retention and see the number or percentage of people who have turned off autorenew already. So, I already know that my churn in this first month is already going to be above 30%. So, I'm going to retain somewhere around 70% of users, likely not higher because 30% have already turned off autorenew.
Jacob Eiting:
That's pretty brutal.
David Barnard:
So by looking at that, I already know I'm going to retain probably less than 70%. So, then you run some quick math on even as that churn declines over time, because month two it'll hopefully be less and month three it'll be less and less, fewer and fewer users will churn and more people will stick around, but the math's not looking good for those two lines to cross.
Jacob Eiting:
I'm interested, David, so just pre-selecting drove that much difference in behavior. How many people on the A still ended up buying monthly? Did it make a significant difference?
David Barnard:
Yeah. Okay. So, here are the numbers. I'll round them up to just make it easy for listeners since we're not going to have a screen share or anything. Of the people who saw the annual as a default, around 800 of them went with annual, but almost 400 selected monthly. So, a lot of people who saw annual did switch to monthly. So, 33% of people chose monthly even though they saw annual as a default. On the opposite end, the people who saw monthly, there were almost 1,100 people and this was an even split, so 1,100 people selected monthly.
So, there was actually a 4.5% increase in people converting to the monthly. So, when they saw the $4, more people did start that free trial than people who saw $40. Side note, I am using Apple's new pricing where it is actually just $40 and that's a whole another experiment I'm actually running currently is the $40 versus $39.
Jacob Eiting:
You are actually running the side-by-sides.
David Barnard:
That's my second experiment.
Jacob Eiting:
We have great content in 2024 because David's running-
David Barnard:
I don't have enough traffic right now. Somebody help me out, let's get some traffic.
Jacob Eiting:
Start driving some ads or something, I don't know.
David Barnard:
But anyway, so it was $40 or $4 a month. That's another thing is I was experimenting with making the price differential not quite as attractive, but that's another experiment I'd like to run is making the monthly a little more expensive to drive annual. But in this case I was like, "I want some monthly revenue." So I wanted it to balance out a little bit. So, I did the $40 and $4, which is not a crazy deal for annual, where typically people who are pushing annual would make it like 50% off, 60% off to do the annual, because they're trying to get that money up front. So, people who saw them monthly as a default, it was 1,100 stuck with the monthly and only 100 switched to annual. So, way more people who saw annual switched to monthly and almost nobody, like 10%.
Jacob Eiting:
Upgraded.
David Barnard:
Less than 10%.
Jacob Eiting:
Yeah, I mean that's probably going to wipe out your results just there, because it sounds like the total number of conversions is similar. It's like 1,100, 1,200, something like this.
David Barnard:
Yes.
Jacob Eiting:
Yeah, yeah, yeah. That's fascinating. So, just priming people on annual is going to make a huge difference. That's interesting. I mean relatively, 40 is not the lowest for weather apps, but it's also some apps are for a year. It's 100. So, it's already on the low end, but that's really interesting.
David Barnard:
So 10% more people converted on the monthly.
Jacob Eiting:
You mean on the annual primed, more people converted on monthly.
David Barnard:
No, people who saw the monthly as a default, more of them converted by 10%. 10% more people became subscribers when they saw $4 versus seeing $40. Again, we can revisit these numbers. Actually, a little shout-out to the community here. I've actually been sharing charts and the actual results from this in the Sub Club Community and there's been a great discussion on that. If you're not already on the Sub Club Community, go to chat.subclub.com and then there's a join link there to join the community.
I shared it in the private area of the community, but yeah, 10% more people converted to become subscribers who saw the $4 instead of the $40. But still, when you start doing the math on the churn, on the monthly and everything else, I'll probably still end up making quite a bit less money on the people who saw monthly and stuck with monthly, because get money upfront versus giving them 12 opportunities to churn.
Jacob Eiting:
Also, you have to consider the time cost of money. That money now is literally worth more value perspective than it is.
Charlie Chapman:
One question to both of you guys, I guess, related to this is when running an experiment like this right around a launch, is that data, not that it's invalid, but isn't that a very different cohort than the people who for the vast majority of the time are going to be finding your app through?
Jacob Eiting:
You're certainly learning more about the preferences of an interesting user base than you are [inaudible 00:54:36]-
Charlie Chapman:
Which is super valuable still.
Jacob Eiting:
... for your unsubscribe rate.
Charlie Chapman:
Especially if you're going for big launches as your strategy, because it's good to know how to optimize those cohorts, but that's going to be different than a month from now all the people who are finding your app through just search.
Jacob Eiting:
Definitely, just the volume you already alluded to David, is as you're running subsequent tests, that's a very subtle test split. I mean, I'm interested that there's any perceivable result there. Actually, I would love to run it for longer to see if that disappeared, because you're talking 5, 6%, that could be noise.
David Barnard:
It was statistically significant though.
Jacob Eiting:
Yeah, I'd be interested to run those tests on, as Charlie was saying, run rate downloads, but you'll have to take bigger swings, because the volume of people going through that funnel is going to be a lot lower. So, to get statistical significance, you don't want to be testing green button, blue button. You want to be testing $1,000 and $3.
Charlie Chapman:
Completely different paywall versus this completely different paywall.
Jacob Eiting:
You're trying to do gradient descent, a machine learning algorithm trains, right? Where you want to just create tests that are big enough apart. You're trying to determine if it's too cheap or too high or whatever. The bigger differences you take, they'll be notionally correct. So, the bigger swings you take, if it's like, "Oh, actually, there's a little more value in the higher side, so maybe don't make that A or that B your actual number," then find a new number that's closer and then test that again as well. It's really awesome, David, that for a limited time only and for a limited amount of effort, you got a paywall set up that you can remotely configure, especially for a developer in your position where you have to loop somebody else in and you've got other bugs and features and things to be working on.
Now, as the business oriented partner, you can pretty much on your own be running these experiments, analyzing the data, and all of that stuff.
David Barnard:
Yeah, I didn't even have to talk to Brock to launch that second experiment. I changed the price on App Store Connect. I already had two SKUs, and so I changed the other annual SKU to $39.99. I left the existing SKU at $40 and I set it all up in RevenueCat to run this subsequent experiment without touching code, without updating the app, without even talking to Brock actually.
Jacob Eiting:
Did you test it at all in a sandbox context or anything like this?
David Barnard:
No.
Jacob Eiting:
Just go for it. YOLO. It's good. I mean, you should be able to. It should be fine, especially if the products are already approved and everything. It's really cool. I think this is one thing on RevenueCat's side that we've just started to cobble all these tools together, but we haven't yet... Talk about RevenueCat, thinking about, "What's a launch for us?" We've been shipping these little features for the last year and a half. When you see them all start to interplay and play together, now we have a thing.
Now it's like having people go through the whole flow. We need to spread the word on how to run these tests all inside RevenueCat, how to design plans, how to do it all in one place. Because I think, like you said, David, 20 years in or whatever, how many years the App Store we're in, you've never done this before. That's crazy because it works, right? So I'm excited to see more and more developers do this and even developers that were already doing it, doing it easier, focusing on other things and things like this. So, anyway, revenuecat.com.
Charlie Chapman:
Before we land this plane, is there anything else with this launch, either things that you're going to try going forward or were there other failures I missed that you wanted to cover?
David Barnard:
Another huge failure was just me getting my expectations up. We talked about it on this podcast about product channel fit and all the different things. It got mentioned in TechCrunch, which is a big deal, and I was in 9to5Mac and MacRumors and Boy Genius Report. It got a lot of attention. Again, if the DMA hadn't happened, maybe there would've been a little more momentum going, but anytime you're launching an app, you got to think like, "Okay, where am I getting attention, and then what's the overlap of the attention I'm getting and my ideal customer profile?"
Then the reality is no matter how much you want to convince yourself otherwise is that of those maybe 30,000 people across all the different press mentions I got, let's say 100,000 people probably saw the headline. Oh, another weather app. I don't care. Even if they are potentially even my ideal customer profile, they already have three weather app subscriptions and they don't even get past the headline because they just don't even care. Then they see the app and they see the video, they see whatever, and it just whittles it down further and further and further.
So, the real overlap of people who actually care enough to go click a link and go buy my stupid little app is so small that even when you can get a lot of attention, again, if that attention isn't a huge overlap of your ideal customer profile, it's just not going to be as impactful as you think. So, back to maybe this whole strategy has played out, unless you're Apple and are launching these massive consumer friendly apps that genuinely everybody is part of your potential customer base, then it's not going to be as impactful as you would hope and assume it would be. So, another big lesson, got my hopes up.
Charlie Chapman:
I think maybe the thing there is it's unreliable. Not necessarily unimpactful. I limped along an update for iOS 17 to get interactive widgets for my app, and it just so happened that for whatever reason, editorial picked it to be in one of their lists. It was the first one in the list and that was just massively impactful. I've had other ones where I've even gotten on lists, but it was lower down or it wasn't one that they pushed as hard or whatever reason and it didn't have those numbers. If you keep swinging on those, some of them will be hits and some of them won't.
Jacob Eiting:
You just got to launch over and over, right? You got to launch over and over.
Charlie Chapman:
Launch, launch, somebody should make a podcast with that name.
Jacob Eiting:
But yeah, especially you have a podcast about launching. That's a great idea.
David Barnard:
For those who don't know, Charlie runs a podcast called Launch where he talks mostly to indie developers about launching and running their app businesses. But yeah, I have it in the notes. Where do I go from here? It's launch, get feedback, iterate, update, get feedback, launch. From here, it's ship, ship, ship and then keep doing these experiments and keep trying new things. Keep taking swings.
Charlie Chapman:
There's the episode title. We don't do that for this one, do we? It should be ship, ship, ship. All right. Well, David, it was a pleasure having you on your own podcast. Thank you for joining yourself. Where can people find you and your work?
Jacob Eiting:
I think we should plug Charlie's work. Where can people find you, Charlie?
Charlie Chapman:
revenuecat.com. I write articles on the vlog. No, I am on all the different social media things. Most places, it's Charlie M. Chapman. On Twitter, which is what I'll still call it, it is _chuckyc, I think is my handle there. Then I have an app called Dark Noise, which is a white noise app, which provides lots of fodder for good RevenueCat content, as well as a podcast-
Jacob Eiting:
We'll have you on after your failed launch.
Charlie Chapman:
Yeah, perfect.
Jacob Eiting:
I guess David, you don't really plug Contrast. So, contrast.co right is where you can find all your apps.
David Barnard:
contrast.co.
Charlie Chapman:
The links of course will all be in the show notes, and I'm actually not the one who has to put them there, so I'm excited about that.
David Barnard:
That's nice. But yeah, anybody else willing to share stories of failed launches and failed product updates and screw-ups and everything else, get in touch, david@revenuecat.com. I want to hear more stories like this. Hopefully, I broke the ice a little, being really open and honest about some of my failures with this launch. The thing is the story's not written. We talked about this launch was a disappointment, but there's still huge opportunity. Hey, I got close to 2,000 people to pay 40 bucks a year for my stupid little weather.
Jacob Eiting:
I don't know where the failed-
David Barnard:
It's pretty cool.
Jacob Eiting:
... came in your head, David, at all. I think that's an unqualified dub in my mind.
David Barnard:
All right. Well, it was super fun talking to you guys. Thanks for having me on.
Charlie Chapman:
Thank you.
David Barnard:
Thanks so much for listening. If you have a minute, please leave a review in your favorite podcast player. You can also stop by chat.subclub.com to join our private community.