I believe Justin's comment opener is the closest thing to getting the best possible response: you have to invite response to reasonably be in a position to expect some. So, I'd think about a logical, visible spot to include a little sentence reminding the new plugin user to provide some qualitative response (i.e. posted, textual feedback) and (especially) imploring the user to also responsibly give feedback on compatibility, with a link to the spot where to select the WP and plugin versions plus the "works" selector.

All this does is essentially drive up the number of compatibility votes. And I believe in the power of statistics; the greater the sample, the bigger the representative value. Put with a sideways smile, it's much harder for a large number of trolls/ignorants/malcontents to be deliberately wrong, amid a much larger body of users generally painting the big picture of user experience.