Proposal to Overhaul the Shortcode API in WordPress Goes Back to the Drawing Board

Robert Chapin, who contributes to WordPress core, published the first draft of a roadmap that explains how the Shortcode API could be overhauled. “The decision to create this roadmap arose from specific needs that are not met by the old code,” Chapin said.

The proposal has an aggressive timeline with development starting in WordPress 4.4 and ending in WordPress 4.7. In WordPress 4.4, a new syntax would be introduced that provides opportunities to make significant changes to the API.

Here are a few examples of shortcodes that use the new syntax.
Self-Closing: [{{shortcode}}]

Attributes: [{{shortcode attr1=”value1″ attr2=’value2′ “value3” ‘value4’ value5}}]

Enclosing: [{{shortcode}$] HTML [${shortcode}}]

Multiple Enclosures: [{{shortcode}$] HTML [${encl2}$] HTML [${encl3}$] HTML [${shortcode}}]

Escaped Code: [!{{shortcode}}]

In the WordPress 4.5 development cycle, the focus would be to deprecate the old syntax, “Plugins that register shortcodes without declaring support for new features will raise debugging errors to alert developers that support for the old shortcode syntax is ending,” Chapin said. Posts using the old syntax would  continue to work.

During the WordPress 4.6 update process, shortcodes using the old syntax would be converted to use the new syntax. The Shortcode API would continue to provide deprecated support for the old syntax to provide a smooth transition.

An important point to note is that the new syntax does not support HTML inside of shortcode attributes. This leaves the potential for many sites to break as shortcodes may not perform the same way prior to WordPress 4.6

The transition process ends with WordPress 4.7 where support for the old syntax is eliminated. Shortcodes and plugins that use the old syntax would stop working. During the WordPress 4.7 update process, a second attempt would be made to upgrade old content to use the new syntax.

The Proposal Raises Concerns

The proposal has drawn constructive criticism from several members of the WordPress community. Nick Haskins, founder of Aesop Interactive, voiced his concern saying the syntax isn’t easier for authors to use and will affect a large number of sites.

Mika Epstein, who voluntarily moderates the support forums and interacts with users on a daily basis, sums up a list of concerns that many developers agree with.

  • All the plugins and themes on the planet we will break (because we will, they won’t read or test). We have to degrade them as gracefully as humanly possibly. Continuing to say “Well the developers were notified and should have updated” now that we’re as big as we are is not sustainable.
  • All the very (legitimately) angry end users who are broken because they didn’t upgrade plugins and themes (or the themes/plugins didn’t get updated). People were rightly angry last time. It’s the end users, not the developers, who will be hardest hit by this change.
  • Communicating clearly to the users that it’s now {{gallery}}. That’s going to be very hard. Incredibly hard. Updating their old posts (keeping in mind Justin’s Markdown caveat and those who use them as an aside – I know I know) is easier than making sure everyone knows what to do. At best we can keep tabs on the ones built into WP and perhaps use the logic we have in the visual editor NOW to convert them, but we have to figure out how to make sure everyone knows. This is nothing like the move of Menus to customizer. That was confusing, but the users could see what happened. This is a legit change, your old way is no longer going to work. That is huge.
  • The number of users who have premium themes and plugins that do not get update alerts. These people are simply not going to know they need to update and this is not their fault. We should never be breaking them if there’s possibly any alternative.
  • Users will be upgraded by their hosts vis-a-vis one-click installs and managed hosting so they will have up to date WP and out of date plugins/themes. So yes, many users will be on 4.7 and then a theme from 2014. It sucks, it’s the reality, we know it’s the reality, we cannot stick our heads in the sand.
  • Plugins that are already using {{template}} tags in their code. Yeah, I’ve seen it. Most of them use it for search/replace within their own code, but we’ll want to make sure we check for everyone in the repo who might be doing it on their own.

The best argument against using the new shortcode syntax is made by Stefano Agiletti. Agiletti says Italian keyboards don’t have keys that create curly braces.

Maybe English people don’t know that { and } are not present in all keyboards directly as the [ and ] are. On Italian keyboards [ and ] are generated using ALT-GR+è or ALT-GR++ and keyboards show ALT-GR basic sign like €@## and [ ].

To get { and } you need to type ALT-GR+SHIFT+è and ALT-GR+SHIFT++. Most people don’t know about this combination (I think only those who write code do) and the parentheses are not written on any key.

Back to the Drawing Board

After a considerable amount of feedback and concerns shared by developers, the proposal is heading back to the drawing board. In a comment summarizing the feedback, Andrew Nacin confirms that the new shortcode syntax does not fit the core team’s vision of being easy and intuitive for non-technical authors to use.

At the same time, he cautions that the team needs to do something, “The proposed syntax significantly clashes with the proposed vision and given all of your feedback, we’re clearly going to have to go back to the drawing board. Please note that we still need to do something, but maybe we can think further outside the box.”

I highly encourage you to read the proposal and the comments that follow it. It’s a great read that highlights how difficult it will be to make changes to the Shortcode API that don’t end up causing a lot of sites to break.


24 responses to “Proposal to Overhaul the Shortcode API in WordPress Goes Back to the Drawing Board”

  1. Aha ! No mentioning of the customizer as you promised last night?

    Seriously though, regarding the proposed API, I am shocked about the reason why a new syntax is needed. To quote them:

    “Our old [ and ] delimiters were easily confused with the way those characters are commonly used in English quotations and sometimes even in URLs.”

    Who uses square brackets for quotations? Are you kidding me? There are so many other things to develop and fix, and it’s shame if not scandalous on how they choose to spend time and energy on things that are not crucial. Prioritizing items means nothing here, the whole thing is run like a big bureaucratic government. And this thing will be spanned over 3-4 versions… is “common sense” a foreign concept here?

    Again, who confuses the [], or uses them as quotations?

    (Moderators, please be gentle on me)!

    • Who uses square brackets for quotations? Are you kidding me?

      No, it’s not a joke. I saw it in use twice today in a quotation. It’s not mentioned in the post but in the comments on the proposal, changing the syntax is also a step towards making the API more secure. There’s a lot of good that came out of the discussion such as highlighting the difficulty of changing the API and the various issues surrounding those changes. I think it generated a great discussion and I’m happy to see the draft was published ahead of time instead of outright implementing the proposal.

      Everyone has a list of items they’d love to have the core team focus on but that’s not how it works.

      P.S. Can you tell me if you received an email notification that says your comment was approved?

      • No I don’t get approval emails Jeff, only when somebody posts, or responses.

        Fine, even if some people incorrectly use [], shouldn’t be that their problem. Instead now we have to spend several months, and break countless websites,just because some use the [] incorrectly.

        All I’m saying is that there a gazillion other things more important to focus on.

        • It was only a proposal of one way to improve the API not a concrete roadmap of what’s going to happen. Therefor, no sites will be broken because nothing in the immediate future is changing.

          • Got it Jeff, thanks.

            The security fixes sounds weird too, all these years and nobody had any security issues with this API, at least not documented. Security should be above all though, so if that’s the case… I can be very understanding.

            Chris Cree has a good point too, by saying the whole idea of shortcodes is to make things easier, so core developer’s keep that in mind if possible, please.

            How about that customizer, huh? Putting some shortcut buttons would be fun ! Take it easy guys, it’s just a joke.

    • One very common usage of square brackets in quotations is when quoting something literally even though it contains some kind of spelling or grammatical error. In that case, [sic] is inserted immediately after the quoted error.

    • I frequently use [ and ] for insertions into quotations. But I still don’t understand the need for this change. It has never once caused a problem.

      So I’m thinking that either there has to be more to this, or else the core team are acting on false information.

    • Square brackets used within quotations (not as quote marks) are quite common and correct in English. I’m not sure about other languages. They’re used to indicate that the quote has been edited to fit with the sentence structure outside of the quote. This usually deals with pronouns. They’re also used to add context that might not be a part of the quote itself.

  2. I like mine and others ideas of using custom HTML tags instead.
    As for keyboards I have to type ALT GR+7 for { or ALTGR + 8 for [. Its a single keypress for < and a shift +. So is 3 keypresses vs 4. {{}} is 8 :).

  3. Personally I rarely use shortcodes myself because I can just code what I need, so maybe I’m not seeing something correctly here. But doesn’t the idea of coming up with a somewhat complex syntax for coding shortcodes kind of defeat their whole purpose? I thought the whole point of shortcodes was to substitute something simple and noncodelike to make it easier for people of modest technical means?

    • I agree about the thinking that shortcodes are meant to be user friendly for non-technial people. So makes me think is part of this step implementing something like the Shortcake (Shortcode UI). Which a lot of plugins and themes already have different button to add shortcodes with a nice simple UI so maybe it checks if it has that if not adds it own?

      All just my own thought and would be inline with how they have been changing items such as galleries, images and video display actual content.

    • >because I can just code what I need

      How do you insert the output of your code in posts or pages? Do you produce plugins or stick the code you write in to theme files?

      • I do both depending on which is more appropriate for the situation. I prefer that so I’m not littering client sites with shortcodes that may end up orphaned one day when (not if) the plugin/theme is eventually replaced.

  4. When I’ve seen the proposed syntax I hoped it was a joke. How can anyone think that this kind of syntax is user friendly? Not to mention that the curly brace like Stefano Agiletti said is a symbol that is not present in many keyboards, and only tech savvy people know how to produce it.
    Please, think twice before making anything that could negatively affect people in non English speaking countries

  5. Usage of [] to show modifications/additions/truncations in quotes is not only not incorrect but mandatory in scholarly publications.

  6. It’s a bit sad seeing the WP trac packed with tickets left to rot for years, and yet those in charge find the time to look for solutions for problems that don’t exist.

  7. It’s good to see that plan got shot down swiftly. I thought using square brackets made a lot of sense, but having looked at my German keyboard right now, I see that I have neither curly braces or square brackets, only keys :/

  8. As a non-technical reader I agree with all the arguments against this proposal. I do not understand why there is the need for a change but, for what it’s worth, here is my input…

    If I could change the Shortcodes API I’d want them to fail well. If I remove, or deactivate, a plugin something like [shortcode_name sc_vars1=”123″ sc_vars2=”something”] is shown publicly all over my website in place of the originally intended content. It looks shabby. And with no in-built method of finding and deleting unhandled shortcodes, it becomes very frustrating trying to manage a site with a decent number of posts.

    I’d wish for text which meets the shortcode syntax, but for which there is no active handler, to be hidden or some alternative (pre-defined) content used to replace it.

    In addition, the ability to easily search for and edit/remove unhandled (and handled) plugin shortcodes would be very welcome.

  9. Wow.
    My gut reaction to this is pretty negative. We would be adding “Stuff That Coders Like”, that will confuse everyone else, make shortcode insertion very prone to errors, make newbie site owners freak out, and probably break lots of stuff out there.

    When I need shortcodes to do fancy or odd stuff, I can code for that behind the scenes. I don’t want to subject normal humans to it. We don’t allow PHP in normal editing spots. It would seem that this is moving in that direction. And as you can see above, readability takes a big hit – it could end up nearly as “fun” as regular expressions. :)

    btw, great point on characters missing in non-English languages. I hope people continue to keep those in mind.

    I am pleased that they’re taking a pause for the cause on this.


  10. First, I think Robert Chapin and others involved in this effort deserve a real round of applause for taking on the increasingly complicated and unsustainable shortcode system now in use. These are the kinds of issues that are must fixes for the longer term, but rarely result in much glory for the effort.

    That said, I really think the solution needs to be a completely new feature that replaces the shortcode system altogether, one that is awesome enough to create incentives for developers and users to drop shortcodes in favor or the much improved alternative, while maintaining support for the current shortcode feature for some defined period.

    I think that would also prevent users from potentially getting confused and suffering errors, developers from pulling their hair(s) out at the thought of being forced (you know devs, they hate being forced to change anything they don’t absolutely have to), and innovate WordPress by a leap in the process. I hope that makes some sense.


Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

%d bloggers like this: