Ideas for Premium

I just listened to Insiders #3, and the opening finding (folks who’ll use Premium get it for free) was painfully ironic. I wanted to suggest a few reflections after the video, and would love to hear your opinions on them too.

TL, DR: Focus on what a new student would want enough to pay for, and what takes relatively little time to arrange.

First off, the video introductions. With all due respect, they’re probably better free introductory tutorials out there, especially in Python. Furthermore, this is only useful to people who prefer video-based learning - are people going to pay 10 dollars a month for that?

It’s important to keep in mind the global audience of Exercism - to someone from the U.S. or Europe, ten dollars is worth a few coffees, for many outside - I’d go so far as to say most - 10 dollars is a lot of money. (In fact, I’d generalize this to offer localization for Premium, it’d probably increase incentive and total revenue. I don’t know if we collect demographic data, but a quick glance at Exercism’s Impact shows that we have a good number of users in countries where 10 USD is significant)

Furthermore, as Jeremy mentioned, it’s these people who actually need our educational resources the most. I think that’s a difficult problem to solve: I’d tentatively suggest financial aid, but that brings in a host of problems.

So in effect: I doubt this move is gonna increase Premium subscribers. I think we should capitalize on our assets, as said in the video - but video based tutorials is hardly our asset.

What are our assets?

The first thing I say about outsiders about Exercism is mentoring + community. I suggest two important changes in mentoring: first, Premium users get supermentors to mentor them alone, in a separate queue - for those unlucky to get a bad experience first time (note that this is more likely as new mentors will flock to the very exercises new mentees do), this may push them off the platform altogether.

Second, a more long term mentoring for new programmers. For regardless of people clearly saying that Exercism is for learning new languages, not programming, a tremendous amount of people use it to learn coding itself. A more long time mentoring would help the student a) gain confidence, b) have someone to turn too in case of issues, and, well, there are a lot of advantages as I see it.

Another thing I consider an asset is the hoard of languages. Perhaps we could have a new role of “Translators” - folks who specifically assist people learning one language from another. I can imagine a demand for such people, but it might be hard to obtain them. Just bouncing ideas here, we can discuss the practicality later.

I’d also link the review idea here. We could expand it - maybe include flashcards or such, to highlight differences between languages for folks who’re translating, or features for new people.

Priority support, perhaps? I’ve long disliked the idea that people who pay get special treatment from the team, but if we’re scrambling for ideas, this is perhaps something to consider. People won’t pay for one or two features (unless they wanted to help Exercism, in which case they’d have donated).

I think, in essence, what I want to say is that none of the current Premium features are useful. I don’t mean to be rude, merely honest. If I could, I wouldn’t pay for Premium if I didn’t want to support Exercism. You’ve got 60 people - perhaps a lot of that are people who just wanted to donate and did so based on the extra marketing? And I repeat: I don’t mean to be rude, I just want to call this out. Premium features should be worth paying for users who aren’t interested in supporting Exercism inherently.

What do you think about these ideas? And do offer your own ones, too!

2 Likes

I appreciate all that. Thank you for being brave and willing to disagree, but in such a respectful manner. A lot of your concerns mimic mine too.

One thing I’d say as a gut instinct to the mentoring ideas is that not enough people request mentoring for me to imagine it would lead to any increase in Premium sales.

It seems to me that one of our main problems is that we don’t know our users.

How might we get to know our users? The only thing that comes to mind right now is to ask them what they care about. Are there large obstacles to a survey on the home page?

( I am deliberately holding off on proposing specific questions and/or designs for the survey. )

That’s an important issue, but mentoring remains one of our strongest and unique plus points. I’d say we should work toward fixing it: which is admittedly tough. However, putting it out as a touted feature would definitely bring more attention to it, I feel. I’d feel much more enthusiastic about “an experienced mentor guiding you in your journey of programming/journey in LangX”, vs. “Have someone review your code for this problem” (current version).

And I really like the survey suggestion! All the same, people are often curiously unwilling to respond to surveys. I’d suggest a badge - just for incentive, if the idea gets of the ground.

Except that mentors are volunteers. We cannot guarantee that people will get better mentoring if they pay.

While I agree with both those things instinctively, the data has shown the opposite.

  • We did add notifications, emails and prompts to people to get their code mentored about 6 months ago. And it made absolutely zero difference.
  • We changed the language from “have someone more experienced mentor you” to “get code review” and the number of people requesting it doubled.

Regardless of this bit in particular, I would be concerned that our supermentors might not want to just mentor the richer people who can afford Premium. Which might in fact make it less likely those people got mentoring.


On the knowing our users bit, I’m not sure I agree that we don’t know our users. We have done various surveys and other things in the past. We know that a proportion of our users are new to coding and that Exercism isn’t the right product for them, and we know that probably 50% of our users come from poorer countries. But there’s also a large proportion (I’d say a strong majority) who are in EU/US and who are professional developers. Our challenge is that we have a very varied user base, who want different things and have different incomes. You can see this from reading introductions on the Discord server for example.

What we don’t know is if people would want to pay for video upgrades or other Premium features. Dark Theme has been the most requested feature for a decade, with probably an order of magnitude more requests/posts/etc than anything else, but only 4 people per day want it enough to sign up for it. I’m pretty certain if we’d asked people whether they’d pay for Dark Theme, they’d say yes based on all the noise over the years, but that doesn’t necessarily then translate in reality.

So while I’m most certainly up for asking our users questions, I’m not sure exactly what answers are going to mean we bank the house on their responses (which is sort of the stage we’re at).

Speaking only for myself, the mentor queue tends to be empty enough that I’m happy to mentor whomever wants mentoring.

This is a thorny problem. I donate monthly because I believe in Exercism’s mission and I want it to survive. There’s no particular premium features or exclusive content that I’m looking for (although dark mode is nice).

Because we want the core functionality to be freely available, we can’t put the web editor or the test runners behind a paywall.

I wonder if there are similarly education-oriented people/orgs on a site like patreon.com, and what kinds of things they offer as exclusive content for patrons.

1 Like

I never doubted we had these statistics. However, at least to me they do not readily suggest what our users want.

I agree that people being mistaken about what they would pay for is a big problem. (Maybe this can be improved through very carefully targeted questioning?)

Out-there idea (beware – off-topic!): have people vote on Exercism development/maintenance direction with their wallet. That is, have users pay up front for the functionality they want. (I haven’t thought this through yet.)

This idiom is unknown to me, and Wiktionary and Google are of no help.

“Bet the house” is a more common variant :slight_smile:

That’s intriguing. Do you feel that’s because people actually are uncomfortable with the idea of “mentoring”? I’ve read the forum post behind this, and it does seem so - or atleast, people consider mentoring to be more serious and thus conflicting with their purpose of learning a language for fun.

Perhaps that’s a vocabulary problem? I’m fairly sure that most people would like help and guidance when they’re new to some thing, but maybe the word “mentor” insinuates a certain degree of responsibility from the student’s side that they don’t want?

I think as a team you decided it was a vocab problem, and that led to the renaming. We could do that here… though I’m not sure what to call it.

By my opening comment, I didn’t mean that 10 USD/month was unaffordable, I meant that it’s not a throw-away amount. I wouldn’t consider it the “rich” people who buy mentoring, rather the dedicated ones. I would definitely be interested in a queue with people more dedicated to their coding journey, and it’d leave me - and them - with a much more rewarding experience. I also think a few (not too sure about the numbers) supermentors have recently expressed dissatisfaction with the mentoring system, a primary issue of which is that they put in a lot of time and effort into sessions that never materialize. That would be much less of a problem with this idea - well, the rich ones still might not be dedicated, but it would decrease it.

I don’t fully get you. If you’re saying that supermentors might not have time for a more in depth mentoring session, I’d disagree, most active supermentors spend a lot of time in non-code-review mentoring anyway. And supermentors are mostly better than normal mentors, so Premium users will almost always get a better experience.

That makes it even more likely that Premium users will be more dedicated vs rich. As such, this is a problem in any form of Premium we have, but we don’t have a choice at this stage.

Have you considered financial aid application? (The current features aren’t important enough, I’m saying for the future)

So what’s your plan: to aim to all of them and possibly lower quality and to aim to a specific group and lower audience? (Not making a suggestion, merely curious)

Well, “pay” is one thing, but I guess 10 USD for dark mode and ChatGPT is more than users bargained for. If we add features which are explicitly related to programming and have a survey with the named price, I think we’d get a more accurate idea.

Isaac is one of those selfless mentors who seems to be perfectly happy to spend time even when the student doesn’t really care, so I’m not too sure whether you’d be enthusiastic about these suggestions :joy:.

Sounds good, but unfortunately I doubt people would reply. “Pre-booking”, as I’d call it, works best when there’s a lot of hype about a product.

And even if users do pay, we risk great disappointment when the product is released and that’s actually not what the payer thought he/she’d get… that’d lead us into murky waters.

For all those who replied, thanks for your time!

This to me is the most intriguing idea to contemplate. There are already books out there along the lines of Java to Kotlin or Python for the Busy Java Developer.

I haven’t thought about it much, but maybe present a solution in one language with tips on how to transpose it into another language, along the lines of our approach articles. Maybe after a solution passes the tests, the student could be provided with a choice of guided transpositions to other languages. Of course, it’s more work, for the infrastructure as well as the content, so it is exhausting to contemplate. But I’m not sure what people would pay extra for beyond more content.

It also just occurred to me about adding new languages. Perhaps the addition of a track could be crowdfunded. Once a certain target is reached, then it can be developed. To do the track exercises would still be free. And some of the revenue could go toward the track implementor/s. (This is not a self-serving suggestion, as I doubt I will ever create a track. :slight_smile: )

I especially like Ruby From Other Languages, which provides the brief start up translation I’d like to see in Exercism.

Nice idea!

Haven’t most popular languages been added? Will people source obscure languages?

My thinking is, if you want an “obscure” language, then pay for it. So, yeah, it depends on how many people want a language that’s missing, and on how much they want it and are willing to pay for it. Of course, there needs to be someone willing to implement it, which is why a financial incentive might be needed for the less popular languages. But it could well be a dud idea.

Great discussion, happy to add a few euro cents…

  1. What do our users want?
    From a PO perspective the most valuable insights come from continously interviewing users: Finding out what challenges they experience when pursuing their goals. Understanding users‘ problems is more important then understanding what solution users think they need.

Picking a niche will make it easier to provide a clear value proposition.

  1. Premium features
    Besides mentoring (which for me is by far #1) I found the syllabus very valuable on my way back to coding. I started with C but it had no syllabus. So I took a language that had one (Python). Mentoring is super-valuable to me because I gain competency faster. I would btw pay (significantly) more for a dedicted mentor who I can also contact for non-Exercism coding quedtions. The syllabus is more a convenience thing. There are a ton of these elsewhere but not with all the Exercism features around it, mentoring but also the fantastic UI.

I think the Exercism model is somewhat similar to free to play games when it come to what someone will pay for and someone else won‘t. The features you can pay for are those that either save you time (tractor vs shovel) or give you fame that you usually only get with experience.

So in a sense, a paid feature for someone experienced lets them skip steps in the syllabus while the free version requires the unexperienced to finish all exercises in loops before moving on to dictionaries.

These are merely guesses of solutions to a problem I‘m not sure I know more of than every user. Does Exercism have a product vision?

Regards
Philipp

P.S.: As it happens almost everyone in this thread mentored me at some point over the past two months- thanks much for your kind support.

Hi Philipp, nice to speak to you again!

I think this supports Matthijs’ suggestion of a survey. @iHiD is willing to put up a survey, but he (rightly IMO) feels we can’t make decisions confidently based on them. What do you say?

The Syllabi is very closely linked to the video tutorials - would you find them useful to an extent you’d pay a decent amount of money for them?

The actual syllabi itself is a volunteer effort and is currently free, so how do you propose to monetize this? (BTW, I think I’ve understood now what @tasx was trying to say: we can’t make users pay for something volunteers give in case the volunteers don’t give. Good point, though I feel unlike syllabi development, which is beta and not very common, we do have supermentors who quite want to help others).

I would disagree - gaming is for enjoyment, while Exercism is for learning. Thus, I don’t think saving time is a primary idea in education - learning better is. Thus, I’d focus on providing more value instead of saving time for the student :slight_smile:. Gaining fame - the Exercism way is rep, but paying for rep kinda corrupts the rep system IMO.

Perhaps I misunderstood you?

Thanks for your contributions to this discussion, and I hope you keep on at it! I think the About page is akin to a product vision.

An idea here: we brand Exercism as a place to get “really good at programming” - might affect our audience? Should we be more explicit about our goal of teaching new languages?

1 Like

A survey is a dangerous path to go down. It will likely give you the information you ask for. Which could be very different to understanding what problems users are trying to solve by using exercism. Structured continous interviews - definitely more work though than a survey.

Re gaming vs coding: I tend to disagree. Games are about acquiring skills and solving problems and that experience of achievement delivers positive emotions. And in ihe end of the day it‘s all about time. If I want to get really good at something and there are two options where one is fast and expensive and the other is slow but free- I take the former, if I can afford it. Not sure if this is adequate but maybe coding tournaments could be valuable for newbies to watch and the pros to compete…?

Re syllabi: Following the aforesaid I‘m absolutely willing to pay for a syllabus as it lowers the entry barrier to embark on a new language. Probably not relevant for pros at some point unless it’s a new paradigm like Haskell, Unison? Actually, I would pay for every language I add to my roster. But again- this is just me. I never coded commercially and am still far from it.

Re pricing: The pricing model is part of the product strategy to execute against the product vision. I like simple models. Like $10/ month all inclusive. And I‘ll be happy to add another $9 to support someone who can only afford $1. And maybe I mentor them, too.

I was comparing it slightly today with freeCodeCamp (non-profit) and Codecademy (ultra for profit). In the case of codecademy they are a for profit but they offer some free basic options and a montlhy pro; what people mostly pay for are the career building (become a data scientist, a full stack developer, cybersecurity analyst interview prep, etc…) they actually have now two options “plus” see more courses, have access to quizzes and the “pro” where you have full access, profesional certifications and carreer guidance- personalized job recomendations. In the case of freeeCodeCamp they also profile it in a similar way like learn this to be a full stack developer but they promote it more like learn to code for free. I have seen they are making several ways to raise money like support the site for 5$ a month, subscribing in youtube, or when they want to create a new course they raise money for that course in general. So I was wondering in the case for a premium version for excercism how long would the average user be using it/paying for It, it would differ from insiders that want to support the site and don’t need extra new features to keep supporting it.

1 Like

@feelhipcamper, sorry, I forgot to reply.

Could you give more detail on this?

Good point - and others have also mentioned that bringing in gaming itself to Exercism would be a feature they’d appreciate enough to pay. What I don’t understand now is how could we make the process “faster”? Apart from a separate queue, but by and large, the mentor wait time is very low already.

Hmm, that’s a nice idea to integrate if financial aid is considered by the team.

Thanks for your contribution!

Fact is, FCC makes way more in donations than Exercism does. The “raise money for a new course track” is a good one, and was previously mentioned in the thread too - but as with everything else, we don’t know if people will actually fund a new track.

Re Structured interviews [Teresa Torres](Continuous Discovery Habits: Discover Products that Create Customer Value and Business Value continuous discovery habits - Google Suche) is worth a read.

Re Faster: I was refering to being mentored vs learning by yourself. I guess you learn from others in a work environment and if you‘re lucky someone is mentoring (if time allows) but it seems to me that being mentored helps you get better faster.