WordPress Core Contributors Call for User Testing on the Menu Customizer Plugin


Ryan Boren published a post to the Make/WordPress Core blog this afternoon, titled Trust, Live Preview, and Menus in the Customizer. In it he clarified the reasons why he and several other core contributors are committed to iterating the customizer and identified the feature as a means of building user trust through live previews.

Being able to make non-destructive changes and preview them is an important component of building that trust. This is perhaps most noticeable in the “save and surprise” approach of the widgets admin screen – every time you add a new widget, modify its settings, or move one around, the changes are saved and appear live on your site, even if you’re not ready yet. The customizer is our framework for live previewing changes. We are committed to providing live preview for all aspects of site customization and making it usable on all devices from phones to large screens.

Boren briefly summarized the history of the customizer and alluded to a few possibilities the framework may offer in the future:

The customizer has come a long way, but it still lacks some features and needs time to mature. We have many improvements planned and in progress, including transactions, partial refresh, theme installation, speedier loading, scaling to large screens, and possibly even integration with front end editing. Our live preview framework offers many possibilities.

The Menu Customizer plugin was tentatively approved for merge during last week’s core development meeting. In order for the it to be officially approved for merge on June 17th, the plugin will need to meet the feature plugin criteria outlined in the core handbook.

“We have eight days to get the Menu Customizer plugin ready for merge,” Boren said. During this time the flow team will be testing and documenting the flow and visuals for the menu customizer.

Boren invited anyone who wants to contribute to this effort to create flow comparisons of the existing flow through Appearance > Menus versus flow through the customizer. This essentially involves walking through the experience of setting up menus, taking screenshots of the flow, and publishing them as a captioned gallery.

“Please help us capture the flows through Appearance > Menus used by you and your clients,” he said. “We need this information to ensure our new interfaces are mindful and aware of how WordPress is actually used.”

Anyone can contribute to WordPress in this way, as it doesn’t require any coding. The core team is looking for people to capture real user scenarios to help in making the final decision.

There is No Timeline for Removing the Appearance > Menus Screen

In the original merge proposal for the Menu Customizer the plugin’s author, Nick Halsey, outlined what he called a “fairly aggressive” plan for the removal of the old menus admin screen. As contributor resources are scarce when it comes to the Menus component, Halsey favored focusing all new development on the UI in the customizer.

The timeline he outlined was for WordPress 4.3 to point the Menus link in the admin bar to Menus in the customizer and later releases (WordPress 4.5 or 4.6) would remove all core links to the Menus admin screen.

WordPress users reacted strongly to this aggressive timeline for removing the old menus screen, but the timeline was merely a suggestion as part of the proposal. Halsey was not keen on merging the plugin without a definitive timeline for removing the old menus, a factor which he considered a “dealbreaker” for merge.

However, WordPress 4.3 release lead Konstantin Obenland confirmed that no official timeline has been set.

Ryan Boren also confirmed that WordPress will continue to maintain the Appearance > Menus screen should the plugin be officially approved for merge in the coming days:

Meanwhile, the Appearance screens will remain and will be maintained. Appearance > Menus recently received some attention in the form of a few fixes. More attention is needed and will be given. There are still differences in the flows each approach best enables, whether it’s new site/theme setup, small maintenance tasks, or dedicated content managers for heavy usage of widgets, menus, or other pieces of content that benefit from having a preview mechanism. We should gather quantifiable metrics when it comes to performance and time to completion for a given flow, as well as evaluating the less-objectively-quantifiable perceived performance. There may come a time where the worlds converge; however, that time is not now.

This confirmation should assuage those whose opposition to the Menu Customizer was solely based on the aggressive timeline proposed for removing the old menus screen.

The great divide on the Menu Customizer revolves around one aspect of improvement that Boren mentioned in his paragraph about the future of the customizer: scaling to large screens. The vast majority of WordPress users and developers who are following this debate are those who would be more likely to configure a menu in a desktop environment and not via mobile (where the customizer is currently designed to shine).

Many who oppose the merge of the plugin have identified the cramped UI as the primary reason that it does not provide a better experience for users. You would be hard pressed to find anyone who is opposed to live previews or better usability on mobile devices.

Those who manage WordPress sites via desktop are not willing to sacrifice the old menus screen for a new UI that currently caters primarily to smaller devices. Until the Menu Customizer can adequately provide a UI that fully adapts to all screen sizes, resistance to the feature is likely to continue.


18 responses to “WordPress Core Contributors Call for User Testing on the Menu Customizer Plugin”

  1. I think this whole mess has (yet again) uncovered two major issues with how we do things in the WordPress community:

    1. While all development is open, unless you make a serious effort there is very little chance you’ll know about what’s happening until it suddenly pops up as a new feature about to ship.
    2. As the community of users grows, wide-scale user testing across all types of users becomes more important.

    For those of us who follow core development and feature plugins with rapt attention, this new feature was not a surprise, but we are in the minority. Furthermore, the people this change will affect the most (the average user) are not aware of this discussion and will not know of the new feature until they come across it in an update.

    I have no problems believing that early user tests have proven the Customizer menu handling to be far superior to the current system. Nor do I have any questions about the claims of critics that a removal of the current feature in favor of the Customizer solution will cause widespread confusion. And this is why I was never in doubt that those at the top of the pyramid would make sure the implementation would not be destructive, as exemplified by Ryan and others.

    The larger implication here is that we need better communication across the board about what’s happening. Yes, some of that responsibility lies at the hands of the users, but the major onus is on us that work on new features. Due to the sheer size of the WordPress user base it is not responsible to spring new features on them, regardless of how well thought out they are. People have an intrinsic fear of anything new, and we have to make sure any time something changes in a fundamental way that we ease them into it. This can be done by providing ample information before the shift happens.

    • The people this change will affect the most (the average user) are not aware of this discussion and will not know of the new feature until they come across it in an update.

      I’m not sure what your point here is. This statement is true of any piece of software. In fact, it’s true of most things in life. Do you expect every one of the tens of millions of WordPress users to follow its development blog? Do you expect that every user of Drupal or Chrome or Firefox or Ubuntu or Nginx or Joomla knows about their upcoming features before they ship? Do you keep up with the development of every piece of software on your computer, server, smartphone, etc? What point are you trying to make?

      • I dunno, Apple does a good job of making people aware of what features are upcoming, whether you’re a fanboy that stalks and speculates about everything, or a casual fan that’ll see it all summarized somewhere. Not sure what point I’m making specifically, except that it’s not unprecedented for the general public to have easier access to upcoming features of something.

        • I’m imagining some sort of Google-like feature where users are proactively told about upcoming changes, ie. when they go to the menu editor or Customizer a flag pops up and says “work is being done to improve this experience. Do you want to try it out?” whereupon the feature plugin is installed.

      • What point are you trying to make?

        I think the point Morten is trying to make is that, as you mentioned (Morten also mentioned the same) many users do not keep up with the development of WordPress, and that perhaps a conversation could be had about raising awareness amongst those users.

        Personally, I don’t think many of those users are going to be interested in the going’s on in the WordPress world. For most people WordPress is just another thing they deal with to get something else done.

        That doesn’t mean a conversation can’t be had, or an effort can’t be made to increase communication and awareness amongst average users. In fact, I’d say the exact opposite. As WordPress grows it’s more important than ever to think about communication strategies that could potentially reach those average users.

        I say this because, I think, to most of those average users it’s not WordPress that’s important to them – it’s those other things they are trying to get done that are important to them. If they start to feel sandbagged by changes and get too pissed off, loyalty to the WordPress project isn’t going to stop them from switching platforms.

        Firefox has been doing some interesting stuff recently to raise awareness around their product, with their Firefox Friends campaign. I’m not sure how effective it’s been, but it’s been cool to watch.

        • I read Morton’s comment and got the feeling that he was touching on an issue that I’ve struggled with at times using WordPress. It’s not about the changes that development brings to the platform that people need to be more aware of, regardless of their overall interest in the software versus their interest in the things the software allows them to do. I don’t think it’s helpful to anyone to pit the process against the product like that.

          We all know that some changes in WordPress affect users at different rates. The recent removal of title text from the link editor is a great example alongside some sort of security or minor bug fix.

          The average WP user can continue about their business without ever knowing that there was a bug or security issue and that said bug or security issue has been fixed. Most didn’t even notice the minor tweaks of the admin color scheme recently, which was more visual and kind of obvious. Those aren’t the kind of updates that are going to cause concern for the average user. And keep in mind that most minor bug and security issues affect a very small percentage of users and does not fit within the definition of average.

          The link editor change was caught by any user that uses links within their posts and pages, and many aren’t likely to follow developmental discussions at any level on the subject. Yet they were forced to change because what they knew was simply gone. It’s those kind of changes that need more noise before implementation. Overall, the link editor update was minor and easily overcome for the average user, but it merely demonstrates a point.

          Personally, I don’t like the Customizer in WordPress. Sarah noted in her article about Ryan saying it should scale to large screens. I develop on a PC so that thing is troublesome for me. I also have the theme editor on Tumblr to compare it to and, while Tumblr hasn’t gotten it right yet, Tumblr is leaps and bounds ahead of WordPress with their live previewing editor.

          A slower timeline makes sense for average users, a spectrum which I consider myself at the high-end of. Having both the Appearance -> Menus screen and the Menu Customizer available for a good long while is probably the smartest way to go since it affords everyone the chance to use it and learn it, WordPress Developers the opportunity to work on the Customizer and make it more appealing/usable for the wider range of users, and ultimately offers all users an option between live preview menu creation and robust menu administration, of which there will be a substantial base of users preferring each option for as long as both options are available.

          At some point it’ll have to come down to the fact that on the whole, users will split between each method of menu management as long as there are two options available. But that should be incentive to convert users from the old to the new with ingenuity and rich features, rather than cutting them off because some feel it’s wasted effort on an obsolete product.

          This probably wasn’t your point, but my take on it is that we need to prioritize the information that average users are exposed to and concentrate efforts toward avenues that maximize exposure, even to the most “cosmopolitan” of WordPress users. One solid and mainstream public debate on the subject would be a great start….

        • Yes. Exactly. WordPress to most people is a means to an end. When it changes, it disrupts their process. Easing them into these changes will lessen the level of frustration.

      • “I’m not sure what your point here is. This statement is true of any piece of software.”

        I tend to agree with you, John. I don’t think this change affects the average user that much at all. To them it’s just a little different way of working.

        It’s developers this affects the most. And as we know, that is a very, very large community.

        Keeping in touch with WP developments should be a priority for devs, but I understand and appreciate that that is quite a challenge.

        Many devs, for starters, have a “day job”, so time is limited.

        Secondly, not only do you have to keep up with WP changes, but devs also need to keep up with changes by other major players (and maintain compatibility with), like WooCommerce.

        And every year WP brings out a new theme that plugin devs need to make sure their plugin works with.

        I speak from a plugin developer pov, but I’m sure it’s just as challenging keeping abreast of relevant development for theme devs too. And I do know, they have the added challenge of keeping up with the latest design trends.

        And then there’s the languages and the latest trends and developments there – PHP, CSS, HTML and JS.

        I can understand perfectly WP devs being so time conflicted that stating intimately connect with WP’s mooted developments is not a priority.

        It’s easy to put our trust and faith in WP to not do anything too radical without proper process, because we kind of have to.

        It kinda reminds me of Hitchhiker’s Guide to the Galaxy regarding the destruction of Earth:

        “There’s no point in acting surprised about it. All the planning charts and demolition orders have been on display at your local planning department in Alpha Centauri for 50 of your Earth years, so you’ve had plenty of time to lodge any formal complaint and it’s far too late to start making a fuss about it now. … What do you mean you’ve never been to Alpha Centauri? Oh, for heaven’s sake, mankind, it’s only four light years away, you know. I’m sorry, but if you can’t be bothered to take an interest in local affairs, that’s your own lookout. ”


    • I follow the chats and things and always wondered; the best place for normal users to be exposed to new features would be where they are, inside their wp-admin. Why not just an update note about a new feature and allowing the user to activate it as part of user body pre-testing. This will warn the users that a change will ultimately happen and give them a chance to play with the feature before it is completely rolled out and replacing what they currently know. Goog does it allot, where you are able to start using the new feature or continue using the classic until sometime in the near future when the classic becomes no longer an option.

      • That’s an intriguing idea.

        I suspect a good amount of that is already done on WordPress.com.

        People on the Automattic side quite often test new features on .com long before the make it to .org.

        MP6 was on .com in early-2013, nearly a full year before it arrived in WordPress 3.8.

        • Yeah, I bet. However, I would think that wordpress.org is the place with more developers that will actually kick the wheels and really test out your changes. On .com; I am guessing the majority of the users don’t really delve heavily into developing their site and making it work with a multitude of plugins.

    • You say:
      For those of us who follow core development and feature plugins with rapt attention, this new feature was not a surprise, but we are in the minority.

      I think this is where the problem is: the balance between contributors and users seems dangerously low.

      Even at the risk of being misunderstood, I would say that WordPress (as a project) should not be too concerned about what the users think.
      WordPress (as a project) is not a company that must satisfy customers, instead it must listen to contributors.

      As contributors are also users, you will still get the general idea of the trends in the so called user community.

      User and contributor are not physical persons, they are roles:
      – the user needs WordPress
      – WordPress needs the contributor

      Let’s go and test the menu customizer

    • Let’s be blunt here: The problem here is that not only has the change been poorly communicated and poorly explained, there is still a major lack of a really good reason for forcing this change in the first place.

      Over the time since this started, I have finally figured out that the issue is most related to using wordpress admin on a smart phone or small screen tablet, with a side order of “we love the customizer!”. It’s perhaps the most classic case of causing the most harm to create limited benefits that might be better in the future.

      The issue then becomes quite simply: What harm is done to desktop / larger screen users in order to make small screen users happy, and what are the percentage of users in each case?

      I just don’t think anyone bothered to work through the full justification case. Would this perhaps not be better being handled by a plug in or special admin setup template for small screen users? Are we talking about fixing something for 10% of users and causing needless change for the other 90%?

      It doesn’t help that the customizer isn’t always the best tool. The customizer widget section, as an example, is a huge backward step from the existing process, at least for desktop users, making a simple job into one that is much more labor intensive. Is this perhaps not the best tool for the job?

      I think a lot of this decision is based on “we love the customizer!” rather than any solid basis of user demand, requirement, or need.


Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

%d bloggers like this: