The underlying problem with the importer is that it doesn't utilize the XML-RPC interface of wordpress. Importing using XML-RPC should solve all the problems related to the export file size and should give better incremental import functionality.

AFAICT this is how the blogger importer works.