Git and WordPress: 3 Tips to Do It Better

petersuhmThis post was contributed by guest author Peter Suhm. Peter is a web developer from the Land of the Danes. He is the creator of WP Pusher and a huge travel addict, bringing his work along with him as he goes.
 


This article is based on a few ideas, thoughts and opinions I have about Git and WordPress. There is still no de-facto standard for how these things are dealt with in WordPress and I think it is our responsibility, as a community, to have discussions about them. These are my opinions and I would love to hear yours as well.

Let’s get right to it.

R-E-S-P-E-C-T

I have ranted about the subject of repository structure before and I have strong opinions about it. In my opinion there is one – and only one – way to structure Git repositories in a WordPress context. That one way is the one-package / one-repository approach. Let me explain why.

While working on WP Pusher, I have seen many different approaches to structuring Git repositories. Two approaches are especially common, and, in my opinion, equally bad.

The first one is Git-it-all, where everything, including WordPress core, plugins and themes are put under version control. The other one, and more common, is wp-content-under-version-control. I say that both of them are equally bad, because no matter what, you are going to have a lot of 3rd party code included in your Git repository, which quickly becomes a mess. Every time WordPress or one of your plugins or themes ships a new update, you will have to somehow deal with it within your version control system.

The argument for having WordPress itself under version control is most often to lock down the version of the code. This seems a bit redundant to me, since WordPress is already developed using version control (SVN) and comes with a version number for each update. Why would you need to manage that yourself when it is already managed for you? Today, installing WordPress with Composer is super easy and then you can control your version from the composer.json file. There are no excuses. Do not include WordPress in your Git repository.

The second approach, wp-content-under-version-control, is often chosen as an easy way of deploying multiple plugins that might depend on each other. Still, I am not buying it. Having 5, 10, or even 20 Git repositories in your wp-content directory is really not an issue. Also, if your plugins depends on each other, take a look at must-use plugins.

So here is the deal: You need to treat each of your packages (be it a plugin or a theme) with respect. You need to give each of them their own Git repository. They are their own entities and needs to be separated. Keeping them locked up together makes it impossible to share code across projects. It is the way Git is supposed to work, it is the way Composer is supposed to work, and it is even the way WordPress itself works.

To Ignore, Or Not To Ignore

Yes, that is the question and I do not have the answer. I am, of course, talking about the .gitignore file. The file that tells Git which files and directories to exclude from version control. Actually, the real question is: Should you precompile your dependencies before shipping your plugin, or not? (I will focus on PHP dependencies, since I do not know enough about front-end things.)

In an ideal world, you would never ship something like Composer dependencies with your package. It would be up to the user to compile and fetch those dependencies that fit their specific environment. That is the whole point of dependency management tools. However, in WordPress, there is absolutely no sane way to handle these PHP dependencies. So if you are shipping a WordPress plugin that relies on 3rd party dependencies, I would say “ship it all”, even though it feels very wrong and awkward.

This is bad, though, because what happens when Mailchimp for WordPress ships with one Composer package in v1.0.0 and WP Pusher ships with the same package but v2.0.0? Oh, the horror. The plugin that is loaded first will get to autoload its classes. The others will not (since it would result in a “class exists” fatal error). I probably do not need to explain why this is bad.

As a WordPress developer, you can try to mitigate this by making sure that the functionality you rely on is actually available at runtime, but this is not optimal. However, I really think this is WordPress’ responsibility. We cannot do PHP without using 3rd party code. That would be stupid. So as I said: No clear answer. I guess it just depends.

Exporting Repositories aka Zippin’

Recently, someone on Reddit proposed that people start to include a .gitattributes file in their repositories, containing export instructions for Git to use when archiving repositories (zipping them). Basically, by specifying it in your .gitattributes file, you can make Git ignore files that you only need for development.

The Reddit thread was specifically targeting authors of packages distributed with Composer, but we can use this trick as well. If you exclude files from exports, they will not appear in the zip archives that you can download from GitHub or Bitbucket. This is nifty, especially if you use a tool like my WP Pusher to move files from Git to WordPress, since it can drastically minimize the size of the archive files. No need to ship files such as assets, that are already compiled, and unit tests to end users.

Here is the .gitattributes file for WP Pusher:

/features       export-ignore
/apache-ci.conf export-ignore
/behat.yml      export-ignore
/circle.yml     export-ignore
/composer.json  export-ignore
/composer.lock  export-ignore

Before I knew about this trick, I manually deleted these files before shipping the plugin. Now it is done automatically.

There you go. Just a few things I want you to think about and consider if you are a WordPress developer using Git. Thanks for reading along.

21 Comments


  1. This article is just so subjective. If you have 5 plugins that you constantly use together, having them in one repo is not bad. Especially if they are plugins that YOU maintain.

    “They are their own entities and needs to be separated. Keeping them locked up together makes it impossible to share code across projects. It is the way Git is supposed to work, it is the way Composer is supposed to work, and it is even the way WordPress itself works.”

    If you use the same 5 plugins across your projects, its more efficient to bundle them and your bundle becomes your “entity”.

    Additionally, I think there’s nothing wrong with having your entire wp-content as a repo. Ideally, you would only have the theme and plugins you’re working with, and media that’s on the site, although you could argue that ‘uploads’ should be it’s own repo.

    Also, WordPress bundles several things that are their own repositories. So the argument that WordPress works that way is moot.

    I also think it ruins innovation to say “You can’t do it that way because that way isn’t the way it’s ‘supposed’ to be done.” How different would life be if we always stayed inside the ‘intended’ lines of something. You can use Git anyway that fits your needs. Adapting your workflow to someone else and their ideals isn’t ideal.

    Use technology to your benefit. Even if others scorn you for it.

    Report


    1. I honestly don’t see why it’s more efficient to bundle them. I don’t see what the gain is. Multiple plugins treated as one entity implies something that should maybe just be in one plugin from the beginning IMO. Uploads should also never be in a Git repository, just as 3rd party software shouldn’t either. It means that every time something gets uploaded or updated, you have to commit it to version control. That seems very wrong to me and I don’t see how it can be more efficient. Version control is not a backup tool – maybe for the codebase, yes -, but not for stuff like uploads etc. That should be kept somewhere else.

      What I mean by “that’s the way WordPress works” is that WordPress treats themes and plugins as their own respective entities.

      I agree with your last point very much, but I think it is also important to discuss these things – even if we don’t agree! Especially in the WordPress community. Even though WordPress has a huge market share and is doing very well, there are still a lot of things WordPress developers are missing out on because parts of the WordPress eco-system is not completely up-to-date with what’s going on in the rest of the PHP world (hint: Composer).

      Report


      1. Instead of bundling everything in one git repo, why not use submodules? If you have a plugin with custom changes, use a fork?

        I get that having 1 repo when building/maintaining a website (for a client) is much easier, but from my experience, the above mentioned solution is much cleaner and does not “pollute” version control.

        Report


      2. I almost made the exact same comment before I read yours, Joachim. +1 for submodules.

        Report


  2. Sarah. Repository management is something I have often called for from WordPress. It should be in core. Will it be? I doubt it. And so we see plugins and themes loading things over and over. Often, to avoid the conflicts, they ship identical versions but with renamed functions, classes etc. Just look how many times PrettyPhoto or FontAwesome gets loaded. Sometimes it is obvious as there is a distinct call to an external asset, but often the code gets rolled into bigger files, never to hbe dequeued. The system should be something like PHP pecl does. There should be a repository or register of all these assets, and WordPress, a plugin, or theme registers within that WordPress install that it needs asset x. The admin should be offered the choice where to source the assets from (local, private cdn, public cdn) and which version. The feature inside the WordPress admin would look like the plugin panel.

    Report


    1. This is one of those tasks that seems conceptually simple but in automation is not. Even in say Windows software I can link a bunch of duplicated functions into an external DLL and unless they are actually called there will be no barking. If they are attempted to be called the compiler will yell to to the ambiguity of function names, method signatures etc. But if I put it into its own namespace, no issues again as I’d need to specify in code the namespaces or add them to resource references.

      While I am not yet any form of WP guru it is essentially no different than say .NET in these respects.

      If assets are brought in via hooks and enqueue’s, there are duplicate assets and assets that are not. Do you want a backend where it shows 250 assets lets say. WP cannot differentiate such things. Is it to do a diff against every asset? Use CRC storing and having retrieve CRC. The moment someone renames a function or puts a prefix on anything, adds even a “/**/” CRC fails.

      Assets may also be hardcoded into the HTML or javascript, CSS or this or that.

      Its really a much larger task that it conceptually seems like and really is more something that should be mitigated at the server not the application.

      People think in WP lands from most of what I have read that using namespaces is to avoid collisions so signatures of data or functions or objects dont need be prefixed. This is true. However, thats not why namespaces simply exist. They exist for logical grouping. So with Namespaces WP could state things like “Coding standards” require the use of namespaces and assets such as bootstrap, Jquery, Modernizr, Font this and external that MUST be be in these namespace. Anything not compliant wont be available in the WP extensions DL area and “These WP functions” in said namespace are what must be used to load or instance those items. So now WP can know, “Yes we already have bootstap.js” loaded. Anything other than that is less effective and really a kludge, it’d be butchering the core code base to take care of a problem that external third party’s cause.

      The only other way conceptually its done is to tell all developers “Do not include any third party support code in your code” and instead those assets need be downloaded in the back end of wordpress thus supporting a plugin lets say. So the plugin need inform WordPress on install of what external assets it needs, jQuery version this or Bootstrap version that etc. If they are already there good. If not then the user need download them from a repository and then they are registered with WP.

      Either way… it negates what is out there existing. It would mean WP core need inform all template and plugin makers of the change and say, “Do it now as if you dont we wont have your stuff available for download”

      Report


  3. That’s my point exactly. Git wasn’t created as a backup system, but why should it not be used as such? It’s easy and efficient.

    Bundling 5 plugins equals exactly 1 clone to install all 5 plugins. Otherwise it would be 5. It also equals 1 pull to update from master.

    I don’t think bundling implies they should be one product at all. For example, let’s say I use WooCommerce, a user management plugin, a gallery plugin, and two other plugins. Putting those 5 plugins in one repo means you only have to manage 1 repo and can install all 5 in one go. And it will make it easier to update all 5 across the different projects you manage (assuming you consistently use the same plugins over multiple projects).

    Report


  4. We take a very different approach with VersionPress which puts nearly everything in the repository, including core files, all the plugins, themes and the “database” (actually a mirror of it in a Git-friendly format). If you’re just a user of such site, this has a major benefit that you only need the repo to reconstruct anything and everything from the site’s history, without the dependency on any external tool or workflow. It is, in a sense, a perfect backup with all the nice version control features added on top of it. The downsides are relatively minor, mainly the larger repo size but it’s usually not that bad and well worth the benefits.

    Another story is if you are also a developer of one or more of the plugins or the theme, and even more so if they are used on more WP sites of yours. In that case I agree that making a separate Git repo for each of them makes sense and we do that e.g. for our blog site which is VersionPress-backed but the theme lives in its own repo.

    These approaches are not mutually exclusive and I think both of them make sense and are useful in their own right. The article seems to focus only on the latter scenario but it’s late at night here and I might have misunderstood something..

    Report


  5. What version control depends on what we’re doing.

    Plugin development, one repo for just the plugin we’re developing.

    Theme development , one repo for just the theme we’re developing.

    Full site development, one repo for the whole site with WP core and /uploads all ignored. We keep all plugins, including those from .org, in the repo. They’re small, it’s handy to roll them back on occasion. This usually encompasses a custom theme, at least a core functionality plugin, and then any other custom plugins we’ve separated from core functionality.

    We don’t use git submodules or subtrees. I’ve tried several times to get them working and they’re were always breaking on me and I decided they are more hassle than they’re worth for me. That could just be me not “doing it right”, but again I judged it simply not worth it.

    What I wish is that years ago WP adopted giving WP core it’s own directory (a la WP Skeleton), it didn’t, it probably never will… but that sure would make full site development under version control a lot neater.

    Report


    1. This is exactly how we do it too and we share the same frustrations as you with submodules!

      Report


  6. I agree with @jb510

    There is no need to be dogmatic about how one should use GitHub.

    However, if you want to share your plugins and themes throught GitHub as an alternative to wordpress.org, you are certainly right.

    If you do have one repository per plugin/theme you can install and update them with
    https://github.com/afragen/github-updater
    https://github.com/franceimage/wp-alt-repositories
    and yours obviously :)

    Thanks for the .gitattributes reminder by the way.

    Report


  7. Git is nicer to use than SVN (in my opinion) but SVN is better for the WordPress repo (sadly). SVN doesn’t keep every commit like Git and therefore you get a “snapshot of the plugin” at the appropriate states. Has anyone ever tried to just download WordPress version 4.1.1 from Github? You endup with 100+mb because you’re getting EVERY version and little change.

    Point is that like it or not your going to have to use both if you plan on uploading your plugin to WordPress.org. Your plugin will be removed if Otto catches people simply pushing up their entire git repo of their plugin to the WordPress plugin repo.

    Also, if your using git as your main source control your plugin will be near impossible to find. Github and other git based repos are just not easily searchable for WordPress plugins.

    Report


  8. In my experience, WordPress sites are simply not friendly with version control. I cannot imagine seeing too much success with repos for content that does not match the database state at the time. If you try to checkout a past version and cannot recreate the database at that point, your site will frequently break.

    The only success I’ve gotten is to track the entire site, uploads and all, which is quite impractical but is relatively easy to deploy, especially with hooks that save the DB in a SQL file on commit and export it on merge.

    A major caveat is that due to the nature of the SQL data, you cannot merge between versions with any real freedom, if at all really, because the SQL data will differ to much and you will almost always have merge conflicts for which there will be no clear answer on satisfying both states.

    Still waiting for the messiah to come along with all this. The article makes a good point about tracking only the content you are actively developing, but it is unrealistic in it having nothing to say about tracking the DB changes. Again, that’s the only way you will be able to checkout any version of the site and have it still work.

    Report


      1. Only problem is that VersionPress lives outside of GitHub, Bitbucket etc. which make it difficult to adopt.

        Report


      2. Really? That would be a huge feature. What about BitBucket?

        You can push to both GitHub and Bitbucket today, no problem there. As a project, we are moving to GitHub soon.

        Report

Comments are closed.