The widget panel definitely needs an overhaul, but since it's been around so long and so many people use it, it's hard to imagine major changes to it that wouldn't receive backlash.

The two key features that are missing are context and classes. I wish I could specify where a widget shows up and any custom classes it has. But like the Menus section, make these advanced options selectable from the Screen Options - the average user probably doesn't care about them.

For page-specific widgets, I often build them using metaboxes. I'd love to see the widget page somewhat integrated into the edit post screen. The ability to see a preview of what the sidebar will look like, have sitewide widgets greyed out, and the ability to add page-specific widgets would be awesome.