“One idea to implement this would be to have a "Roll Dice" tag, which when the post is submitted, would generate random number(s) for each tag placed in the post to determine success or failure, along with some special markup or some other mechanism to discourage shenanigans.”

Why not use some part of the unique ID of the post, like with NNF. I won't know what this post's ID will be until I've posted it. The previous post by theraje has an ID of #12vskt7akvvk, since this is base-36 (0-9 / A-Z), one could use some simple pattern to decide what the roll was, like "the first number in the ID", or the first letter A-T (20 letters).