Help Test the New WordPress Importer Plugin in Development

photo credit: Ryan McGuire
photo credit: Ryan McGuire

The official WordPress Importer plugin is in shambles and has frustrated users with inconsistent performance for years. The plugin has more than a million active installations and is critical for migrations, unless you are familiar with using more advanced tools. With years of unresolved support threads identifying the plugin’s many bugs, the importer is in need of some help.

In a volunteer-driven open source community, important projects like the WordPress importer can go neglected for a long time until someone gets inspired to take on the responsibility. The good news, however, is that the importer is in the process of being completely rewritten thanks to Ryan McCue, who is also a leader on the WP REST API project.

The WordPress Importer Redux project is available on GitHub with fixes for some of the previous plugin’s most troublesome issues. McCue is rewriting the importer with the goals of making it “fast, lightweight, and consistent.” He summarized the major improvements that are part of version 2 in a post on Make/WordPress core blog:

  • Way less memory usage: Testing shows memory usage to import a 41MB WXR file is down from 132MB to 19MB (less than half the actual file size!). This means no more splitting files just to get them to import.
  • Faster parser: By using a streaming XML parser, we process data as we go, which is much more scalable than the current approach. Content can begin being imported as soon as the file is read, rather than waiting for pre-processing.
  • Resumable parsing: By storing more in the database instead of variables, we can quit and resume imports on-the-go.
  • Partial imports: Rethinking the deduplication approach allows better partial imports, such as when you’re updating a production site from staging.
  • Better CLI: Treating the CLI as a first-class citizen means a better experience for those doing imports on a daily basis, and better code quality and reusability.

Until recently, the new importer had no UI, so the only way to test it was to run it via WP-CLI. A new Admin UI branch of the project is now available so you can download the plugin as a zip and activate it inside WordPress. McCue tweeted a sneak peak at the progress bar for the import summary:

He is also working on a better user mapping UI for the importer. This image shows the UI in progress with the old one at the bottom for reference.

importer-v2

If you want to test the new importer, McCue encourages users to compare the experience to the old importer and take note of what works and what breaks. Feedback is welcome in the project’s GitHub issues queue on the designated general feedback thread.

19 Comments


  1. can’t wait to try it out now that there is a UI thanks for the post! :)

    Report


  2. I installed it and it gave me a PHP error. Had to go into FTP and delete it. BUMMER.

    Report


      1. Actually, the new plugin code depends on the old plugin’s class named against WP_Import() at line #33 of `plugin.php` file.

        So, one needs to install the old WordPress Importer plugin before trying to install the new one.

        I think a note about this would go long way :)

        Report


      2. EDIT: This is how it is.

        Just figured the master and admin-ui branches are different and we don’t need the old plugin. Though the master branch does.

        So, if you are downloading via the master branch zip link, then you need to have the old WordPress Importer plugin installed.

        But if you are testing the new `admin-ui` branch, then you don’t need the old WP importere. For downloading a zip for this `admin-ui` branch click this zip link.

        Report


    1. Me too; an immediate 500 error. This 4.5 site is running on a PHP7/LetsEncrypt/NginX/ hybrid server, so I am not surprised. But its definitely not ready for showtime.

      Report


      1. Correct, it’s still in development, and very far off being released. :) Make sure you use the admin-ui branch if you want to test it.

        Report


  3. This is great news. I just added importing test data to my WP-CLI script. There were so many warnings and errors that I had to remove it. Now I can add that back in. Thank you Ryan and contributors.

    Report


  4. I think the link is not the correct, i was getting a fatal error.I downloaded again from github and it works.

    Report


    1. The link was pointing to the master branch, which didn’t yet have the new changes merged in. It’s now merged, so that link should work. :)

      Report


    1. +1 to that, this has been a long, long time coming!

      Report


    2. Indeed. A huge amount of +1 to Ryan for this. Nothing but respect.

      Report


    3. This is a ground-up rewrite. Basically, the way that the existing importer works is by reading the whole file in, turning that into a giant object of data, then importing that. (This is XML “push” parsing, because the parser pushes all the data to you at once.)

      The rewrite instead works by opening (but not reading) the file, then moving through a piece at a time. (This is XML “pull” parsing, because you specifically pull pieces from the parser as you need.) This means you never actually need the file in memory, just enough to work on. This also means you can pause and resume parsing, as you just need to keep track of where you’re up to.

      This uses less memory, because you don’t need the whole file in memory at once (and the usage is also more constant, rather than dependent on file size), as well as being slightly faster (mostly because there’s less memory usage).

      The GSoC project tackled this from the UI end, which takes the file and changes how the import runs, but doesn’t change the internals. It’s kind of similar to how splitting WXR files works (IIRC), but doesn’t fix the core issues. The new importer is resumable though, so this should be a lot easier. :)

      Report


      1. Also: the running process through the new UI here works differently, in that the browser is responsible for running it via calls to admin-ajax. That’s not a perfect solution, and I do plan on looking into cron running eventually. :)

        Report


      2. This sounds fantastic, Ryan. Do you think it’s ready for heavy-duty testing on enormous imports? I’d love to try to break it.

        Report


      3. It’s specifically designed for large imports (that’s mostly what I’m working with), but the UI isn’t ready for those yet. (Specifically: I haven’t finished the resumable parsing functionality that’s needed for large imports via the UI.)

        It should definitely work via wp-cli though, as there’s no time limit on that, so it doesn’t need the resumable bits.

        Report


  5. Just read this post after pulling out the little hair I have left from countless failed imports. Trying to upload the wptest.io test data to a WP 4.5 installation, either locally with XAMPP or online with my host. Trying this out now and reporting back ASAP with the result. Soooooooooo glad I have found this post and an alternative to the piece of $à!&% that wordpress.org calls an importer. Thank you so so much, you have no idea how glad I am to see a light at the end of the tunnel. I just hope it is not a train coming my way, fingers crossed. Go devs!

    Report


  6. Testing it right now to import a humongous old blog with a bazillion media attachments into WP Multisite. If this works, it is going to save me a world of hurt.

    Report

Comments are closed.