It's time to bring this home. What does a tabletop RPG need to be now?
- Online: Your customers go to your site, where you link to the System Reference Document you maintain. That document is where all the rules and mechanics that you use in the content that they buy is found, and because you maintain one master document you never need to make new editions. Your game is evergreen in every way that matters, and people LOVE stability in products that they use- and you don't need the cost or overhead of a print publishing operation or a physical retail hustle to make this happen, so don't.
- Simple: Your design needs to something a prospect needs to come into cold (no preparation) and stupid (no prior knowledge or experience), and go from sitting down to playing in five minutes or less. That prospect needs to be engaged within 10 minutes, and well on their way to familiarity within an hour. "Character sheet" means a page out of a notepad or spiral-bound notebook that you use in school, not a custom-made form that rivals IRS tax returns for complexity. Embracing liminality makes this happen.
- Regular Content Production: This is what makes you the money. Publish playable content early and often, putting it out there where customers actually shop online: Amazon. Give away the rules; sell the content- and sell it cheap so it's an evergreen impulse buy (That means $5 or less, usually $3 or less.) This is a core way you adapt the Galaxy's Edge model to serve your purposes. The ideal is something new every month.
- Strong Support for Game Masters: This is going to be something you spend a lot of time doing on the site, on your Twitch and YouTube channels, and so on. You do this, and you will increase your business by making prospects confident in their ability to run the game- not just play it. In time, you can get into the business of curating the best fan-made stuff; the more that people play your game, the more you can find ways to sell them content that they will want to buy.
If this can be summarized as "Original or Basic D&D's product size and complexity, but regular support in the form of convenience products and free skill development." then you're on the right track. Cultivate a play culture that supports open table gameplay, no time commitment (which is what boardgames and videogames use to compete favorably over tabletop RPGs right now), and otherwise actively recreate the original gameplay paradigm wherein tabletop RPGs facilitated their explosive growth. Replicate the conditions for success, and you will re-experience that success.
This is what it has to be now: digital, persistent, convenient, cheap, and simple. Get in and playing in the time it takes to boot up a PS4 and sdownload a game to the drive, or get dumped for alternatives that do just that.