#105 – Pascal Birchler on Revolutionizing Image and Video Processing Within WordPress

[00:00:00] Nathan Wrigley: Welcome to the Jukebox podcast from WP Tavern. My name is Nathan Wrigley.

Jukebox is a podcast which is dedicated to all things WordPress. The people, the events, the plugins, the blocks, the themes, and in this case, new ways to interact with your media within WordPress.

If you’d like to subscribe to the podcast, you can do that by searching for WP Tavern in your podcast player of choice, or by going to WPTavern.com forward slash feed forward slash podcast, and you can copy that URL into most podcast players.

If you have a topic that you’d like us to feature on the podcast, I’m keen to hear from you and hopefully get you, or your idea, featured on the show. Head to WPTavern.com forward slash contact forward slash jukebox, and use the form there.

So on the podcast today, we have Pascal Birchler. Pascal is a longstanding WordPress community member, and Core committer, with a focus on internationalization, WP-CLI, and performance.

He’s been working on a project for the past six months to enhance the media capabilities of the WordPress block editor. His work includes client side image optimization, image compression, video enhancements, and speech to text features such as subtitle generation.

It’s unlike anything that I’ve ever seen, you upload media into the block editor and it’s manipulated right there inside the editor, by the browser, not by some remote server.

This is all achieved by harnessing the power of WebAssembly, and we discover what Pascal has managed to achieve thus far, and how he thinks that there’s going to be so much more to come.

We talk about how you can change images from legacy formats, like JPEG and GIF, into other, more modern formats, such as WebP and AVIF. Pascal explains how this is actually achieved, and why the new format so desirable. It’s all about reducing file sizes, server resource consumption, and the speed boost your site will gain as a result.

We also get into the conversion of animated GIFs into videos, and creating videos with transcripts on the fly. And remember, this is all done in the block editor, with the browser itself as the workhorse.

Pascal seems to be onto something here, and it’s really compelling listening to him argue that something like his plugin might be worthy of inclusion into WordPress Core at some point.

He explained some other projects, which he’s been working on inspired by the high volume of feedback he received after releasing his plugin.

If you’re interested in pushing the boundaries of user experience, performance optimization, and compliance with evolving web standards this episode is for you.

If you want to find out more, you can find all of the links in the show notes by heading to WPTavern.com forward slash podcast, where you’ll find all the other episodes as well. And so without further delay, I bring you Pascal Birchler.

I am joined on the podcast today by Pascal Birchler. Hello, Pascal.

[00:03:42] Pascal Birchler: Hello. Hi there.

[00:03:43] Nathan Wrigley: Very, very nice to have you with us. We’re going to talk today, well, I think it’s broadly, we’re going to be talking about images. But there’s an awful lot of different technologies that Pascal has been working on over the, well, I don’t know how long. But I’ve only recently caught sight of the projects that you’ve been working on.

Pascal, just before we begin that, would you just tell us a little bit about yourself? Perhaps what your day job is, what your relationship is with WordPress, and so on. Just a little bio time really.

[00:04:10] Pascal Birchler: Sure, yeah. So my name is Pascal, and I’ve been a WordPress community member for, I don’t know, 15 years now or so. So I’ve been doing a lot of contributions around internationalisation, so like anything multilingual. But recently also performance, and day-to-day, that’s really what I’m focusing on.

So I started as a WordPress contributor. Then I joined Google, as part of a team that is trying to make the web faster, that includes making WordPress faster.

[00:04:36] Nathan Wrigley: Yeah, the performance team have been doing great work over the last, well, I’m going to say about 18 months it feels like they’ve been in existence. But WordPress itself has been made significantly quicker, over the various iterations of the project recently.

But I know that the browsers themselves have become better at doing an awful lot of things. And so I think what we’re going to talk about today will probably involve conversations around what the browsers can handle, and things like that.

But it’s very rare, Pascal, that I watch something online to do with WordPress, or technology, and I am actually having to lift my jaw away from the floor. But a little video that Pascal posted onto Twitter, not that long ago, maybe 10 days ago or something like that, really did make my jaw hit the floor. Because you were showing things that you’ve been working on that I had never seen before.

Not just that, I had no intuition that it was even possible. It’s going to make your WordPress life much better in the future, let’s hope.

Do you want to tell us a little bit about the project that you’ve been working on? And then we’ll get into the weeds of all of the different things that it can do, and yeah, just tell us how long you’ve been working on it, and what it really does.

[00:05:42] Pascal Birchler: Sure. Absolutely. First of all, that’s exactly the reaction that I hoped to get from people, as I’ve been working on this project, or these media experiments, how I would like to call them, for maybe half a year or so. Like I occasionally started working on it and didn’t stop, and start again.

It’s basically a collection of things that enhance the media capabilities of WordPress, or specifically the block editor. Most of the things are around like client side image optimisation, image compression. But also anything related to videos or speech to text, like subtitle generation.

[00:06:18] Nathan Wrigley: So you just swept it to one side, really, how impressive it is. But the key component there, that was so amazing to me, was the fact that it’s all done client side. So perhaps, dear listener, the best thing to do would be to pause this episode, go and search for it over on WP Tavern, and watch from the links, the two little Twitter videos that Pascal put together, and you’ll be in a much better position.

But what you’re going to see is truly remarkable. So it’s block editor based, so you’re in the Gutenberg editor. I don’t know if it’ll work in any other scenario of WordPress, with page builders or what have you. But that’s the paradigm that we’re talking about.

And Pascal is sitting there, and he decides that he wants to have an image which is currently a jpeg, and he doesn’t want it to be a jpeg. He wants to reduce the size of it, so that it consumes less resources on the server, but also can be served up to the end user. He wants to pass Core Web Vitals, because he’s a good Googler.

And so he throws it into Gutenberg, and Gutenberg seemingly converts it. Now I’m obviously doing it injustice. Please tell me what’s happening, because I haven’t the faintest idea how you’ve achieved this. So you put an image directly onto the Gutenberg canvas, and it’s no longer a jpeg. It’s been converted in some way, in the background, disconnected from the internet. This could be on a computer, completely disconnected from the internet. So it’s happening in the browser. How is it happening?

[00:07:40] Pascal Birchler: So first of all, browsers are extremely capable of doing image compression, or even image conversion. But they’re also a little bit limited in like to what extent they can do that. So to overcome those limits, I’m using a technology called WebAssembly. Which allows basically like regular software that you would otherwise install on a computer, and bring that to the browser.

And in this case, this is an image processing library that I’m using. For example, I’m using the VIPS, or libvips image library. And using WebAssembly I can do all sorts of image rotation, thumbnail generation, directly in the browser. And then from there I will upload it to WordPress.

[00:08:19] Nathan Wrigley: So is this, therefore a plugin, that you have pre-installed into the version of WordPress that you were using for the demo there. So this is a plugin that you’ve built, you’ve installed it, and I don’t if there are any settings in there. But even if you just activate it, as soon as you begin throwing images, there is the option to convert them to different formats, right out of the box. You don’t have to do anything else.

[00:08:42] Pascal Birchler: Correct. It’s a plugin that I started. It works without any additional configuration, but of course, if you prefer not changing the image format, but just want to compress images. Or if you prefer, let’s say WebP or AVIF over jpeg, you can say so as well. And there are a few more settings that I added, but mostly for like testing and like, trying different things out.

[00:09:04] Nathan Wrigley: Yeah. I mean, it’s a first run at this project, so maybe we’ll see some things in the future. We’re recording this at the end of 2023, so it’s still really hot off the press.

But just to digress a little bit, and move away from the technology. Why have you done this? What is the purpose of it? Now I’m guessing that there’s maybe some sort of environmental impact here. There’s the cost of storage, and all of those kind of things. What’s the point? Why would you even want to convert images from one format to another? After all, it looks the same, broadly, once you’re viewing it. What’s the purpose?

[00:09:36] Pascal Birchler: Doing this has many advantages. So one is that if you want to do this, but you’re limited in server resources, now this allows anyone to use this technology and compress their images. And compressing images, or using a different file format, as you say, has like good impact on file size. Which means less bandwidth used, has a nice side effect of like images loading faster on your website, which is great for passing all these Core Web Vitals.

But it’s also a user experience benefit. Your visitors will see the images faster. And I think that’s the nice thing about this project, is that it has some performance aspects, but in the end it’s a lot about user experience.

[00:10:18] Nathan Wrigley: So the modern versions of image formats, it may be that some of the listeners aren’t really familiar with those. I’m pretty sure that if you’ve been on the internet for any length of time, you’re familiar with jpegs, and pngs, and GIFs, or GIFs, however you wish to pronounce it. But you’re trying, well, in the video at least, you are converting them into an image format, which maybe people haven’t heard of before.

It’s called AVIF, but also I think you mentioned WebP as well. I know it’s maybe not in your wheelhouse directly, but can you just give us an idea of what’s going on there? What kind of reduction could we have? Let’s say if we took a, I don’t know, a two megabyte jpeg and pushed it through to be an AVIF. Roughly speaking, and I know it’s dependent upon what is in the image, but broadly speaking, what kind of benefits can you get in terms of percentages that are cut away?

[00:11:04] Pascal Birchler: Oh, that’s a great question. I think in my test video, I saw a reduction of like maybe 40% or so in file size.

[00:11:11] Nathan Wrigley: Good grief. Right.

[00:11:12] Pascal Birchler: And there was like no noticeable differences to the eye. But there are like quality settings that you can change. Depending on the level of quality that you like for your use case, you can get it even further down.

[00:11:24] Nathan Wrigley: So really significant amounts of data. You know, if your website is image heavy, then doing this, and there’s no noticeable loss to the viewer. Perhaps there is some sort of degradation, if you were to expand it, and you were a professional photographer, who knows? But something in the region of a 40% decrease.

Do you know if, at this point, so again we’re in December 2023. Is it possible for all browsers to be able to show these images back to us? So it’d be fabulous to be able to use AVIFs, but then obviously if a significant proportion of the internet out there can’t actually see them, because their browsers don’t support that, that would be troublesome. So do you have any inclination as to what the compatibility is with browsers, displaying WebPs and AVIFs?

[00:12:08] Pascal Birchler: I think last I checked, WebP is supported by all browsers. So that includes Chrome, Firefox, Safari, Edge, and so on. And AVIF is not currently supported by default in Edge. All the other browsers support it. And in Edge, it needs to be enabled with a flag.

[00:12:25] Nathan Wrigley: Oh, so there is support, it just merely isn’t enabled. So if you’re an Edge user, you can make it available.

So really there’s no reason not to do this if pretty much every browser is able to support this, why wouldn’t you do this? And I guess the beauty of your system that you’ve built, is that there’s no, there’s literally no thing to do. You just put the image onto the page, or the post, or whatever it may be, in the same way that you would always do, and it just happens on the fly.

Is there a little bit of a wait there? So let’s say, again, my image of two megabytes, something like that, jpeg uploaded. Is this happening more or less instantly in a typical modern computer, or do I have to sit there and twiddle my thumbs for a little while, as I might have done in the past if I was using a server side technology to do this?

[00:13:13] Pascal Birchler: It’s pretty much instantly, but only because you don’t see what’s going on in the background. So when you drop the image in the editor, you can already see it, you can already, you know, add some gradients in the block editor, or continue writing your post. And in the background, the image is compressed, and all the thumbnails are being generated, and finally uploaded to WordPress. But you can already continue editing.

[00:13:37] Nathan Wrigley: Okay. So really you can just carry on with your day, keep editing as normal, and in the background this will happen.

So what’s the flow in terms of where this is getting saved? So, again, we’ll use my example, I’m using a jpeg. Will the jpeg be saved in the media library, ultimately, as well? Or will just the converted image get kept? Because I know that a lot of the other server side solutions, some commercial solutions and what have you, which compress images, they have loads of options and many of them are, for example, keep the original. We’ll convert it for you, we’ll compress it for you, but we’ll also keep the original. And it may not be that that’s what your plugin does. So how does it handle the original? Where does it put that?

[00:14:15] Pascal Birchler: Right now it keeps uploading the original to WordPress, in case you want to restore that or use that for, I don’t know, downloads or something. But this can be made optional, because everyone has different needs. For example, photographers prefer to have the full size images available, just in case, maybe for their clients. But this can be turned off if you prefer to save resources.

[00:14:37] Nathan Wrigley: Okay. So that, I guess, brings me to the next question. What are the settings that you have at the moment? So, you’ve mentioned that you know, you can keep the original, or delete the original, or what have you. Are you able to specify, okay, every time I upload a jpeg, turn it into an AVIF, every time I upload a WebP, turn it into an AVIF? Are there different settings in the backend there that I can configure, so that it just meets my exact requirements?

[00:14:58] Pascal Birchler: You pretty much nailed it. Right now the setting is limited, like you can choose your preferred format, and it uses that. But the next step is to provide a setting per file type, so that you can really specify what you want to be for every type of file that you upload.

The settings are really fine grained in that regard. I hope them to be fine grained. There are also more settings, for example, for thumbnail generation. You can choose whether the thumbnail should be generated on the server, or in the browser as well, which is the default.

And the third option is to have the client side thumbnail generation, but make it smarter. So for example, if you have a photo of a person somewhere, right now WordPress just crops like these 150 by 150 thumbnails from the center of the image. But with the smart option, it will focus on the person’s face to be in the center, just so much more useful.

[00:15:50] Nathan Wrigley: Wait, it makes a decision about what to crop, based upon the content of the picture. So it’s intelligent enough to know that there’s an object over here which is of significance, crop around that. That’s great.

How would that work for other things? This is going to go off in a strange direction. Let’s say I had a, I don’t know, a cat or something like that in the picture, but there was nothing else. There was just this singular cat, and it was top right. Is it able to intuit that there’s an object in this picture which needs focus?

[00:16:16] Pascal Birchler: Yeah, so the image library that I’m using tries to detect the most prominent object in a photo. So there could be an animal, or something else. Of course it’s not perfect when it’s, I don’t know, a photo of like a group of people or something, but it’s better than nothing.

[00:16:30] Nathan Wrigley: Yeah, it has to make a decision about which person is more important in that picture. But again, just to emphasise, if you’re listening to this, we’re not talking about this data leaving your computer, getting crunched on somebody else’s machine, and then being sent back. This is all happening inside the browser, right before your eyes, or not before your eyes. It’s really just happening, and the result comes out before your eyes.

You may not know the answer to this, I hope that you do. But, why do browsers have all of this technology baked into them? What’s the reason for that? I mean, I could guess what some of the reasons would be, but I’m interested to know if you’ve got more concrete data on that. Why would a browser ship with all of these image conversion, formatting options, all the other things you’ve described?

[00:17:14] Pascal Birchler: Well, as I said, the browser does have some image formatting or conversion options, but they’re limited. That’s why I’m able to use external libraries, through Web Assembly. And the reason that Web Assembly is a thing is it just, it makes the web so much more powerful. It allows things like Photoshop to run on the web, which previously wasn’t possible. I think for the web to still matter in years to come, browsers and websites and web technologies need to evolve, support use cases like that.

[00:17:41] Nathan Wrigley: Yeah, I have this intuition, I could be wrong. But I have this intuition that in a decade from now, more and more of the things that I’m still holding out on my computer at home, will have gone over to the browser. And it feels like video and images are still in the domain of the desktop computer often, just because of, well, the amount of data that has to fly around for a start.

But also, if you’ve got an hour long video and you’re trying to edit little bits, it does seem that online solutions don’t quite make the grade yet boy, it’s changing quickly. And if we stare into our crystal ball, a year into the future, two years, five years, I feel that a lot of this will have been taken over by online tools.

So we talked about the fact that you can upload an image, but it goes a lot further than that, because you can take a bunch of images. Now I don’t know how much you’ve battle tested this, whether or not you’ve put in 500, 1,000, 10,000 images. But in the example you used, I think you put three images in, and it did the same thing.

So if you were a photographer, or you had a post that was heavy with images, this can all be done. You don’t have to do one at a time, wait for it to happen. It’ll just put them all into a queue, I’m guessing, and tackle them one at a time.

[00:18:43] Pascal Birchler: That’s exactly right. Yeah, I have this sort of queuing system that I built. It can do multiple images at a time, or only one. That depends on a lot of factors. For example, like how many CPU cores that you have on your machine.

The good thing is that everything happens in the background in so-called Web Workers. And these are separate threads from the main website, which means even if you throw more than one image at it, it will be separate from the main thread, so you’re not blocked from writing any more content. I don’t have 10,000 images on my computer to test it with, but maybe I should.

[00:19:15] Nathan Wrigley: But it will cue them, and so we’re expecting it to work in the normal way. But certainly for a normal amount of images, it’s going to handle it correctly. Let’s say though, that I did throw 100 images at it, which is going to consume a little bit of time. I’m imagining, you know, there would be some human noticeable amount of time for that to happen.

And then I was to stray away from that post. Close it down, for example, or go to a different screen inside a WordPress. Does the browser continue that operation? What is the connection? How is it bound? Does the post need to be open for all of this to happen? And also, what about just failure? If something fails to happen, do we get a notification to say, whoops, it didn’t do what it was anticipating to do?

[00:19:55] Pascal Birchler: So yes, you definitely need to stay in the editor while the upload is in progress. There will be a warning if you try to navigate away. Of course, if this were like a single page application, where you’re not actually like changing origins, going to another website that could continue to upload. But for the upload to finish, you need to stay in the editor.

If an upload errors, there will be notifications of course. Thanks to this queuing system, it’s even possible to change your mind and cancel uploads, if you prefer. For example, if you notice, oh dang, I uploaded wrong image. You don’t have to wait for the upload to finish to fix that mistake, so you can just cancel it.

[00:20:29] Nathan Wrigley: Yeah, that’s great. Now, it’s not just images though, it will handle a whole load of other things. And there’s a couple that you demonstrate. The first one is something which is, obviously something that irritates you a little bit. Animated GIFs or GIFs, however you wish to pronounce them. You’d rather that they were made into videos, and so it will do that as well.

If you were to throw an animated GIF into it, it will recycle that as an actual image, with a play and stop button, so that you’ve got some agency about what’s actually happening on the screen there. Again, so that’s presumably just for GIFs. I can’t think of any other image format off the top of my head, which has that animation possibility. But that will create a video of the same dimensions or, what’s it doing there?

[00:21:11] Pascal Birchler: So yeah, animated GIFs or GIFs are painful. WebP also supports animation, but it’s not that common. But for animated GIFs, they’re really bad for, not just performance, but also user experience, because you can’t stop them. They just autoplay, they’re super heavy in file size usually. And if you, for example, prefer reduced motion, which is like a setting in most operating systems and browsers. A GIF won’t stop. But a video can be stopped, but it can also be preloaded, can have a poster image. So it’s just way better from a user experience.

And the plugin converts GIF or GIF to a video with the same size. Creates a poster image as well. So if the video is stopped, or still being loaded, you will see a preview image as well. You can have either the controls to play, pause, or it just loops and autoplays, like a GIF. You don’t see the difference.

[00:21:59] Nathan Wrigley: It’s what we need. Agency over animated GIFs, that’s for sure. And it just all happens seamlessly. But again, another jaw off the floor moment was when I saw that you were creating video content inside the post editor. So, you know, imagine, dear listener, that you are in a post, you’ve clicked add new post, and there you are.

Shoot video from the webcam, or I’m guessing any camera that might be attached to the machine. Shoot it, save it into the media library inside the post. As if that wasn’t enough, also transcribe it in real time. Now, I don’t know if that transcription gets burned onto the video, or if it’s just something that’s happening as you are watching it.

But that’s truly remarkable as well. And it leads me to all sorts of thoughts about being able to, I don’t know, communicate with other people, with synchronous video, a bit like we have with Zoom, or something like that, inside of a post. I’m not entirely sure why you would wish to do that, but those kind of possibilities becoming available. How did you do that? How does that work?

[00:23:05] Pascal Birchler: You bring something interesting up, because there’s like this collaboration feature being worked on in the block editor. So just imagine having a video call, and maybe saving that video afterwards, who knows?

So for this particular feature I’m again, using the powers of Web Assembly to use the FFmpeg video library in the browser. So FFmpeg is a really powerful tool set. It can do video conversion, all sort of things. And I use that in a browser to basically record the video, or take the recorded video and convert it to like a MP4, whatever is a good format for you.

And for the subtitles, I’m using a different solution, which is based on an open source machine learning models, that do transcription, like speech to text. And that generates a vtt captions file. Similar to srt, it’s a separate file, it’s not burned into the video. And you can use other tools to then edit the captions, or you could even build a caption editor directly in WordPress.

[00:24:04] Nathan Wrigley: Most video solutions allow you to upload and combine the two, don’t they? So it’s got timestamping in, so it knows when a particular word is about to appear on the screen. And so it can handle all of that. But it creates that file on the fly. So it not only creates the video file, stores that into the media library, but if you have the option set for the vtt file, it will also save that. And presumably, because of the time that they were saved, there’s a very high probability they’ll end up right next to each other in the media library. So you could associate the two.

That is truly impressive. I can only imagine what could be possible with that. Because all you are doing is showing that it can be done, and it can be saved right there. But yeah, I kind of feel that once that gets out into the wild, other developers will have intuitions as to what to do with that, aside from you, and may well wish to do something.

But you’re right, you know, can you imagine a scenario where, I don’t know, a graphic designer wants to talk to another graphic designer about a particular post, or a particular page that they’re working on and, well, let’s just hop in, and actually work on it together whilst we’re in the page. We don’t need another third party solution. I don’t know, maybe that’s a bit extreme. But absolutely fascinating how you’ve managed to pull all of this off.

You mentioned the libraries that you were using there to do the video transcoding. Again, is all of that bundled in with the plugin, or is any of this going out of our computer, out of our browser somewhere, and getting converted, and then being sent back? Or is it all happening on our machine?

[00:25:27] Pascal Birchler: It’s all happening on your machine. I know it’s sometimes crazy to believe, but yeah.

[00:25:32] Nathan Wrigley: I mean, really, truly crazy. One thing that you own that I don’t own is you have an iPhone, which it turns out also has an unusual, well not unusual because there’s a lot of iPhones out there. But it’s not a web friendly image format. I didn’t even know what it was called until you showed it on the screen, it’s HEIC or HEIF. And it’s simply for Apple devices.

So I’m guessing if you’ve got a Mac or an iPhone, it will show it just fine. But if you’re trying to upload that to the media library, you’re going to have a problem. So conversion has to happen there. And you’ve got a nice, another nice little option there. If you throw that in, that will also get converted, again to your preferred one. So it could be, I don’t know, AVIF or WebP, whatever you’d like.

[00:26:13] Pascal Birchler: Yeah. So it’s a super annoying format to use on the web. Right now, it will convert to WebP or AVIF, whatever you have set. The file format itself also supports like multiple frames, because on iPhone, you have this feature called live photos, where the photo is sort of like a video. So you could technically even convert this image to a video when you upload it. So you have this live photo effect also on the web.

[00:26:36] Nathan Wrigley: But it’ll do all of that out of the box. So all of that is absolutely fascinating, and we’ll get to something else that you showed in a slightly different video in a moment. The purpose of this, obviously, you’re scratching your own itch, you’ve done something which will definitely be helpful in the future. It does strike me that a lot of this is definitely within the scope of being put into core.

Now, I know that an awful lot of stuff comes through the news outlets each and every week about amazing WordPress products, and some of it clearly is really out of the scope of core. But do you have an intuition that some of this really could be baked into core, would be useful in core? Is that something you are going to be trying to push for in 2024?

[00:27:19] Pascal Birchler: Well based on the reactions of people to the demos I’ve shared, there’s definitely interest. Seems to be useful for many people. Be that the HEIC conversion or the thumbnail generation. So I definitely see potential for this to be in core, and it’s something that I want to strive for in the next year. But it will be definitely limited in scope. Like, you won’t see fancy video recording or something in core, I would say. So I would want to start small, focus on like an M.V.P. solution, and bring that into core and then we can go from there.

[00:27:51] Nathan Wrigley: Yeah. That’ll be an interesting thing to see. One last thing, just before we move on to your upload an image with your friend idea. Another thing which you do is, and this is curious because we had a chat about this before we clicked record. If you upload an image and that image is going to take a few moments, an opaque, hazy version of the image is shown. And it almost looks like you’re squinting your eye or something, you know, it’s gone a little bit blurred.

But the technical side of how you’ve done that is really interesting, because I was anticipating it was just another image, but it’s not, there’s some really interesting work going in the background there, and gradients and CSS. Just explain what’s going on whilst we have the image load. You show us something, which kind of looks like a faded out version of the image. What’s happening?

[00:28:33] Pascal Birchler: So the itch that I wanted to scratch there, is that the experience when you have to wait for images to load is really not great. Let’s say if you’re on a 3G connection on a train, or something. So what I wanted to do is have like a blurred version of the image showing. So as soon as you open the page you immediately see something. You know, oh, there’s an image that might be loaded here, and it looks interesting.

And what I’m using for that is a technology, or an algorithm called BlurHash, which generates these blurry placeholders. And usually to display that placeholder on the website, you would need JavaScript. But I’m converting this blurriness to simple CSS gradients instead. So that when you load the page, you’re not seeing another image while the main image is loaded, it’s just all CSS gradients.

[00:29:18] Nathan Wrigley: So no JavaScript in sight, it’s just CSS, which obviously has its benefits. So upon successful completion of that image, what are you doing with the CSS that you’ve created there? Where is that data going when it’s no longer needed? Do you just expunge it and get rid of it?

[00:29:33] Pascal Birchler: Well, you will still need it for like the next page load, right? So it’s always there, it’s always in the database. And it’s super small, like even for a larger image. The amount of gradients is not that big, so it doesn’t really impact performance.

[00:29:47] Nathan Wrigley: Yeah it’s a really nice experience, because it just gives you, well, you’ve all been there where an image hasn’t even begun to load, and you get all this cumulative layout shift where things are being pushed around, and buttons go further down and everything moves. It kind of gets rid of that, because right away you get this block, which is going to be the same size and shape as the image, which will eventually load.

I think in most cases these days, certainly in the part of the world where I live, we’re really not faced with those problems too much these days. But as you said, if you’re in a place where there’s a slower network or a, let’s say a 3G network or something like that, then having that as an option is great. So yeah, fascinating solution to that problem, I really like that.

Now, I think you probably got a little bit of interest, and I don’t know if the second Twitter video came along after the first one that I saw. So we’ve just discussed what I believe is probably the first one, and we’ll discuss the second one. And I’m hoping, Pascal, in the days and weeks to come, we’ve got Christmas in the way, I realise that. But in the days and weeks to come we’re going to see many more such videos.

You’ve got this really interesting idea of being able to upload images into a post, with a link, or a QR code, which I guess is a link of a different kind. But you could send it to a friend, who could then upload an image into a post, without being logged into the WordPress website. Where did that need arise? What was the scenario that you were thinking, okay, I need people to upload images into my posts, which they don’t have access to edit, and so on? What’s going on there? It’s fascinating.

[00:31:13] Pascal Birchler: I have to admit, I did not come up with this idea actually. So after I published the first demo on Twitter, John Blackburn, I actually opened an issue on GitHub and he was like, hey, you know, on apps like eBay or online banking apps, for example. There’s the option to upload your receipts or something from your phone. So why not make that possible in WordPress? And I was like, oh, that’s interesting. So I built it.

It’s really cool because, I think the scenario is, you’re editing a blog post, you want to add this nice picture and you realise, oh, it’s on the phone. Maybe it’s on your friend’s phone, or like colleague’s phone. So you can scan the QR code, or send the link for that code to someone else. And then they’re able to upload the image to WordPress. And after that is done, and you’re still in the editor, so once they’re done uploading the image, you’ll immediately see the image in your editor, you can continue editing.

[00:32:04] Nathan Wrigley: So you wouldn’t need to refresh that post and come back to it, it would just appear. So let’s say, for example, that I send you that link, and I’ve got my post, it’s half completed, but I just know that I need that image to finish it off. And I know that you, Pascal, have got your hands on it. I could send you the link. You click on the link, select the image in question, and there it is, right inside my post, immediately without a need for a refresh or anything.

That’s so great. That’s just such a curious thing. But now I get the point, and I guess the idea of being able to send it to somebody else is quite interesting. But the idea, certainly for me, you are right, there’s a lot which is held on my phone, there’s a lot which is held on other computers. I have a detachable hard disc over there, and quite often the computer that it’s attached to is, you know, in no way connected to this one. So being able to send myself another link over there, that’s really curious.

It feels to me, Pascal, as if you are just getting your feet wet here. You’ve just got yourself all excited about images and what you can do with them inside of WordPress. So that’s the next question. Do you have any intuition as to what’s coming next? What fun projects you might work on.

[00:33:09] Pascal Birchler: Well, you know, just talking about this with you, gives me a lot of ideas. And sharing these demos with people also sparks ideas in them. And you mentioned that holidays are coming up, so maybe, I don’t know, over the holidays I’ll build something new. We’ll see.

[00:33:22] Nathan Wrigley: Well, I would very much like to see the outcome of your endeavors. Yeah, I’ve definitely got a few ideas of what I would like. For example, being able to record a podcast directly inside the editor. I know that, obviously, video and audio are very similar, it may well already be possible with everything that you’ve done.

But the ability to do things like that inside the post editor, and not have to save it elsewhere, would be kind of interesting. You know, what we’re doing right now, we’re using a third party SaaS app to record this. I can’t see any reason why all of this couldn’t potentially happen inside the block editor as well.

Anyway, exciting times. And Pascal, I do thank you for piquing my curiosity. It really was quite a moment when I saw that video.

If anybody wants to contact you, if anybody has been intrigued by what you’ve said, and they want to reach out, where are the best places? We know that you’re on Twitter, but tell us what your handle is, and where else we can find you.

[00:34:10] Pascal Birchler: So my Twitter handle, or WordPress.org, everywhere else is swissspidy with three S’s in the middle. That’s the best place to reach me.

[00:34:19] Nathan Wrigley: Alright. Well, thank you so much. And Pascal, I really appreciate you coming on the podcast and chatting to us today. Thank you so much.

[00:34:25] Pascal Birchler: Thanks for having me.