Cancel

Justin Tadlock

Gutenberg Can Tackle the Problems the Fields API Tried to Solve

The Fields API.

Never heard of it? That’s OK. Outside of the inner development community, it is not widely known. The average WordPress user does not need to know about it. Before understanding how the Fields API fits into Gutenberg’s future, you must first understand what it is and the problems it was meant to fix.

The Fields API was a proposed solution to one of WordPress’ biggest problems: to build form fields in the admin and save data from those fields, developers need to know multiple APIs, depending on the specific admin screen.

Want to build a plugin settings screen? Use the Settings API.

Need some theme options? Build them with the Customize API.

Have some fields to output on the user screen? Here are two hooks and a mess of HTML table markup; sorry, no official API.

Those are just a few examples, but the truth of it comes down to this: to show something as basic as a text field to end-users, WordPress developers need to know how to do this in a variety of ways based on competing or even missing APIs.

There are historical reasons for this. New features were bolted on top of WordPress over time. In the mad rush to continue shipping features with each major update, few people stepped back and asked the fundamental question about the technical debt that would pile up over the past 16 years. Shipping end-user features helped the platform grow, but developers had to learn all-new functions and methods each time.

Adding to the technical burden, when the Gutenberg project launched, it introduced a new system in a different programming language.

The Fields API would have created a standardized system for outputting form fields and saving field data. It would work with all the existing admin screens and any new features added in the future. Developers could learn a single system and be able to build plugins that worked with practically any area of WordPress.

In 2014, Scott Kingsley Clark took over the Metadata UI Project. The initial idea was to create an API for adding custom fields (meta box fields) on the post-editing screen. Eventually, Clark and those working on the project realized the problem that needed solving was larger than meta boxes. WordPress needed an API that worked across the board. After a year, the project was relaunched as the Fields API.

After years of working on the code behind the project, Clark became burned out. He stepped down as the project’s lead in 2018. With no buy-in from the decision-makers for the WordPress project, there was little hope of it making it into core. At that point, the project was all but dead.

Gutenberg’s development was in full swing. Developers were gearing up for relearning how to add the same basic text fields and other form elements in whole new ways.

The Fields API, had it made it into WordPress before the block editor, could have alleviated the need for developers to learn a new system. However, that’s not where we’re at today. The Fields API never made it past the gatekeepers, and developers have one more thing to stay knowledgeable on.

The question is: how do we address this going forward?

How the Gutenberg Project Can Solve the Fields API Problem

What many don’t understand is that the Gutenberg project is larger than the content editor. The first iteration, Phase 1, of the project was to create a new editing experience. Phase 2 will create new admin screens for site editing using the same components for the editor. Custom text fields, select dropdowns, color options, or one of many other field types all run through the same reusable, component-based system.

That sounds remarkably similar to the Fields API. At the end of the day, the Fields API is simply a standardized method of reusing components to output form fields and save data, regardless of the screen in WordPress.

WordPress needs to be rebuilt from the ground up. Gutenberg provides us the opportunity to rewrite every admin page in WordPress using a standardized system for handling form fields.

From a technical standpoint, Gutenberg has dozens of components. These include a text control, button, checkbox, and much more. It covers the majority of use cases plugin and theme authors need for form fields. These things are not tied directly to the block system. They are simply components that can be used anywhere.

The next step would be setting the foundational layer for other admin screens. It will not be easy. There will be backward-compatibility mountains that the Fields API could have climbed for us years ago.

Given WordPress’ history, developers will likely continue using competing APIs for fields on various admin pages. And, if we’re still at that point in five years, the Gutenberg project will have failed for not going as far as it could have.

For success, the Gutenberg project needs to have a wider vision and a longer-term roadmap that addresses the issues of fields on every screen. Otherwise, projects with easier-to-learn APIs will be more enticing to developers.

The idea of Gutenberg-ifying the entirety of the WordPress admin will be off-putting to many, but WordPress has to solve its form fields issue at some point. It might as well reuse the components that will be seeing active development for years to come.

Category:
Tags: ,
27

27 responses to “Gutenberg Can Tackle the Problems the Fields API Tried to Solve”

  2. Gutenberg solves one huge problem that Custom Fields have, but at the same time, it negates 2 huge problems that Custom Fields had already solved… let me explain:

    WordPress when used as an app. like a real estate site for example that uses Custom Fields, can have front end forms and parametric search pages, so an end user for example can find a house in Los Angeles, that has 4 bedrooms, and it is priced between $500,000 and $750,000. If this was done with Gutenberg today, we could not have a form on the front end for agents to list their properties, and the end users won’t be able to make complex queries to find their perfect house to move into.

    That said, as it is with Custom Fields, these kind of websites are not easy to scale as each field takes 2 rows in the table for each post. As an example if the app. is using 50 custom fields for each listing, and let’s say that we have only 5,000 property listings, we would have a total of 5,000 rows for the listings (posts), and 2x50x5,000 = 500,000 rows for the custom fields for a grand total of 505,000 total database records (rows). Now if Gutenberg can solve the front end forms and parametric search issues and we can use only Gutenberg to create such sites, the database would only have 5,000 total records as oppose to 505,000 !!!

    As long as these issues are not solved, Matt’s dream of total market domination and making WP the operating system of the web is only a pipe dream and not a realistic goal at all. And my understand is that having front end editing or solving the parametric searches are not even in the radar of confronting this issues with Gutenberg, therefore WordPress will stay as a very good but an “incomplete” or not perfect solution to create large, real money making websites. Perfect for small, to medium sites, but never for large ones….

    Create forum topic from comment

    • Saving arbitrary data fields related to a database row is a notoriously tricky problem with databases. You can either modify the database schema on the fly or use EAV (Entity-Attribute-Value) schema as WordPress does, which leads to a bloated database table.

      MySQL 5.7 and onwards supports JSON data type which perhaps could be used instead but I’m not sure that WordPress is going to bump up their database requirements anytime soon.

      Create forum topic from comment

      • to search for parameters stored in json encoded strings inside the post content?

        This would be the worst possible approach.

        Maybe related: Ticket #3739 in Gutenberg github, “WordPress search, unexpected results due to Gutenberg serialization markup”

        Create forum topic from comment

      • Thanks for your reply Otto. However, and I create my blocks with ACF Pro, that is not possible. I directly asked the ACF developers, and the answer I got from them was that at least with ACF Pro, ACF custom fields and ACF Blocks cannot “communicate”. I hope this will change in the future…

        That said, I still believe, as far as keeping the database as small as possible, Gutenberg blocks is the way to go, and as far as using WP as an app. (real estate sites, etc…), I would create block/s for most of the information I want to collect, and create custom fields in meta boxes for the fields I would need for parametric search queries, thus minimizing the database damage as much as possible.

        However, and nobody has commented on this yet, using Gutenberg blocks, we completely lose the option of front end post submissions/editing.

        It started about 10 years ago, when WP really took off and really left the competition miles behind, because by then, WP could be used as a serious professional CMS, and even as apps. thanks to custom fields and the introduction of pages, not just blogs. Unfortunately it has never jumped over the hurdle of really handling large websites at least without spending thousands on hosting to handle the data. As it is now, there is no way (at least at with reasonable hosting fees) sites like Walmart, Target, Amazon, etc… can use WP to sell their tens of thousands of products. Magento can, but not WP. And I think if WP really wants to be the Internet’s OS, the front end posting and the parametric search issues must be addressed. Looking at some of the developer’s websites, and templates, unfortunately the emphasis is still on blogging – I think, the emphasis should be beyond blogging…

        Create forum topic from comment

    • I’ve never used ACF and cannot speak to it’s functionality.

      However, nothing about the block editor prevents front end submissions or editing. You know we have front end posting on the make sites using o2, and it works with the block editor just fine. We don’t have the block editor on the front end, but it’s perfectly possible to do that.

      Create forum topic from comment

      • By saying “nothing about the block editor prevents front end submissions or editing”, you might as well say “nothing about my kitchen chemicals prevents me from finding a cure for cancer”. Simply said, for what I’m talking about, it is NOT possible with strictly using Gutenberg blocks, and o2 does not even come close for an acceptable solution.

        Gutenberg is released for 14 months now. CAN ANYONE name me a plugin or a theme that without using Custom Fields, and using only blocks can achieve a Real Estate Website, a Classifieds website, an Auto dealership website, where a real estate agent can list properties on the front end, and at the same time a site visitor can make a complex query? Such themes/plugins do not exist, and gee, I wonder why? However, there are dozens such themes/plugins that do utilize Custom Fields, BUT by doing so, the database will grow exponentially in almost no time, rendering WordPress unsustainable for LARGE websites – and that’s all I’m saying. Once again, the developers are creating another half baked product… Even Gutenberg as a page builder is a half baked product !

        Create forum topic from comment

  4. An interesting proposition made possible by Gutenberg is to have all metadata saved and managed through blocks on the blog post itself (instead of saving it on some different place, such as a custom table), and then to retrieve and filter the data through an API, either REST or GraphQL.

    For instance, this REST endpoint extracts the metadata from a specific post, and in the process removes also all the HTML contained in the metadata, so it can also power an app.

    This endpoint extracts all block metadata from all posts. We can also filter it. For instance, this endpoint extracts all the Youtube videos from several posts, and it can also specify to extract the URL property only and nothing else.

    To achieve this, I created a plugin, which exports the block metadata in a medium-agnostic format, and a REST+GraphQL API to access it and filter it.

    Create forum topic from comment

  6. What WordPress needs is also a proper ORM that can generate required tables etc. Everything in Posts table is dumb now and it was dumb then.

    There are few things WordPress needs to be a better solution for many use cases. However the main drive today seems to be create a pretty somewhat usable interface to compete with Squarespace.

    Create forum topic from comment

    • I disagree. As a developer I find the post table to only be about 10-15% ‘dumb’. I regularly utilize the vast majority of fields, especially with custom post types.

      There is very little I would change with the WordPress database. I understand some people have concerns, like the gentleman above who mentioned 5000 entries in the posts table could means 500,000 rows in the postmeta table. Running an advanced query on postmeta that is searching multiple ‘key’ fields with a corresponding value can result in a slow query. But there are ways to mitigate that slow down and turn those slow queries into speed demons.

      From a structural standpoint I think it’s the best possible system for the different tasks it needs to accomplish. Keep in mind that the real power of WordPress is the versatility. It’s definitely possible to layout a “better” database for a specific purpose, but WordPress isn’t a single purpose tool. It has to be flexible and cover unknown use cases. Just because you or I don’t need a specific field doesn’t mean that field isn’t vital for someone else.

      Create forum topic from comment

  8. From a “content as a page” perspective, this post rings true. From a “content as a data source” perspective, a fields API would be useful. Gutenberg dumps all of its data to html code, so individual bits can’t be stripped out and made accessible to a REST API call. You can’t get a value of ‘single block’ within GB content.

    Where I work, we use WP as a CMS foundation for both Web and Apps. This means Gutenberg can be useful for managing web content, but you still need fields for specific application data, because specific data and types need to be consumed.

    ACF fills this need just fine, and easily integrates with the REST API, so no major complaints from me. It just feels like WP is disregarding users who want to use the platform as an advanced CMS in addition to creating a website.

    Last I checked, I am not wrong here, but please correct me if I am :)

    Create forum topic from comment

    • Hi Shaun, sorry but I think you’re wrong :)

      If you check my comment above, I show that you can actually extract the metadata from the Gutenberg blocks, down to a single block from any post.

      Of course the idea is not necessarily to use your blog post as the single source of truth of all your data. Sometimes it will not make any sense, such as storing references across different entities in the database, for which you may want to rely on ACF (or something else). But in other cases it can perfectly make sense.

      For instance, if you want to retrieve the collection of all Youtube videos embedded in your blog… instead of duplicating this information somewhere else, since you already have this info in the Gutenberg blocks in your posts, you can already extract it from there…

      Create forum topic from comment

  9. If WordPress is going to continue to grow, we need to think about what the long-term approach should be. I’d wager it’d be things like:

    Continue the modular trend that Gutenberg started, eventually evolving into what is essentially modular components and a GraphQL API.
    The above eventually evolves into provide WordPress in two flavors: 1) the ability to use only the components you want like the editor, user management, media, etc. for advanced use cases or 2) use the wholesale CMS like today.

    This give the flexibility to use it as something other than a monolithic app while still serving those who do (most WordPress users). It’s the same project, just more JAMstack-y so I can use what I need if I’m a developer building a more complex website or app.

    This keeps WordPress relevant as websites becomes less relevant–every part of WordPress could be a best-in-class component. For example, why does every email newsletter app, website builder, business card maker, brochure designer, etc. have its own crappy editor implementation instead of using WordPress’ Gutenberg component? This could be true for every other WordPress component. This would also greatly accelerate the number of contributors to WordPress. …imagine the next new hot startup using the X and Y WordPress components and contributing back to them.

    I’m not claiming the above implementation is right, but like Justin said we need a wider long-term vision of where we want to be, then work our way there refactoring piece by piece.

    Create forum topic from comment

  10. The idea on using Gutenberg for all kind of form fields in WordPress is amazing. It truly is. The problem is when will it available? As Justin said in the article, 5 years consider a fail.

    As the Meta Box’s author (a custom fields plugin) and a follower to the Fields API, I feel the Fields API is more ambitious than what’s said in the article. It shares some similarity with the Gutenberg. But it’s deeply about the data, which is more about the storage and structure. Gutenberg is more focusing on the content (which is HTML & text) at the moment, and it has little support for custom data storage & structure.

    I hope this can change in the future. But after the 2nd and 3rd phases, do we have enough time to get to that point within 5 years? I doubt that.

    Create forum topic from comment

    • The problem is when will it available?

      It should have been available at rollout to core. Unfortunately Gutenberg was included way too early in core. It should have stayed a plugin until all phases would have at least be defined in a very detailed roadmap including APIs defined and at least basically implemented.

      With the current approach again and again breaking changes will be rolled out and plugins and themes have to adjust again and again and support many versions as not all users will upgrade always to latest core immediately.

      Too bad that nobody seems to have seen this coming. :-(

      Create forum topic from comment

You may also like

Newsletter

Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.