The Internet changed everything. You can profit from this or be destroyed.
The publishing world, across all sectors, has been and continues to be fundamentally transformed. Tabletop RPGs are not exempt, yet those currently running the show still adhere to practices with their roots in a pre-Internet paradigm where shelf space and being in print matter most. This is over.
How do you do this now? Well, if you're one of the existing behemoths you can lumber along as you were for a while. But you're not one of them. You need to get with how things are now if you want to still be alive after those beasts finally succumb and die.
- Write The Master Document: This is your System Reference Document. You will put this online. You will let users read that for free. You will link to this document in every product you publish hereafter, taking advantage of ebook technology to link from a page in a book to an external document. You will link directly to the rule or mechanic you're referring to, making is easy and very convenient for users to look up that stuff as they go- and eliminate the need to publish a rulebook entirely. You will update this document when you must, but in doing so you will forever avoid the damage that new editions do to tabletop RPGs; you get the benefit without the cost- and instead enjoy a stronger brand over time.
- Launch Your Site: You will, in time, have two sites, actually. One is strictly for the SRD, so traffic to it doesn't wreck your front page. One is for your publishing efforts; this is how Paizo handles their SRD and it works great for them. Build up to this. At first, you will likely have to go with a free blogging site or free webhost and make the SRD a subpage linked at the blog's main page; you will need to shift when your traffic warrants a full site and a separate one for the SRD, and you should take that as a clear sign of success.
- Love the E-Reader: You're going to publish at Amazon. That's where the folks go shopping now. Learn to love MOBI and make good use of it. PDF is a stopgap measure, and the potentials you're hunting after want something they can read on their Kindles- not just PDF readers on their PCs. If you have something that hits really big, then you can offer a print version via Print On Demand; don't bother otherwise, unless it's a map.
- Impulse Buy Is Best Buy: You're selling what you do sell cheap. Don't be like the idiots in the traditional publishing world trying in vain to kill ebooks with stupid pricing models to encourage print purchases. The real indie publishing successes rarely go past $5, and many are $3 or less- with occasional giveaways when a new book in a series drops. Folks will not think twice of buying your stuff if it's cheaper than a run to McDonald's, especially if they automatically get a de facto free rulebook with every purchase. Kill barriers to entry, and customers flock to your door.
- Online Is Evergreen: Digital publishing is inherently evergreen. There is no warehousing cost, so there is no reason to not have what you publish persistently available for sale. While some categories of playable content are more popular and desirable than others, just about anything you publish is evergreen when you do it digitally and leave print as a secondary avenue. The flipside is that you need to take care that you're putting out there something you still want out there years from now, because Out Of Print does not exist.
There's no reason to go about this blindly. There is someone to use as a model to build from, and that's the guy behind Basic Fantasy. He's doing a lot of this, while remaining focused on a print-centered model; he's doing this as a self-sustaining hobby, but you can see how you can go from hobby to business from there with ease- and that's not accounting for things like Roll20 or Fantasy Grounds.
But how do you go from here to there? There's now a proven path, but it comes from the indie SF field: the Galaxy's Edge story of success. I'll get on about that tomorrow.