I think one of the main issues here is WordPress.org and SVN. Think about it, what if SVN was dropped, and instead all the plugins were integrated with Github. Since all the plugins are GPL anyway, there would be no problems with the movement. Since the .org plugin sites would look active since the Github repo is active.

Here're some more integration ideas:
* Github has a "release" feature, we can take the latest release there as a stable release version of plugins. Clicking on the download button in WordPress.org would simply get the latest release version.
* A button called "development branch" (or something cooler) can also be added beside the normal download button in WordPress.org. That would install the files straight from the dev files instead of the stable releases.
* The last commit messages can be used as the changelog between plugin versions
* Github's issues tab can be used as the support tab in the plugin page.
* Readme.md can be used instead of readme.txt if it's present.

I don't know about the legalities and how hard the .org can be transitioned to this. But for me the ideas I gave above are more than enough to consider doing this.