soledad penadés

A while ago, I posted about this whole business of iPhones saving images in a very space-efficient format which is also a faff, as it isn't supported by browsers or services in general. So each time I want to upload pictures to my food blog, I might get HEIC or JPG images. Then I have to export the HEIC images as JPG, while the JPGs can stay as they are.

I don't know about you, but these things break my "flow".

Yet I am also very patient, and I understand that sometimes technology takes a bit of time to "settle", so I sort of put up with it, hoping someone would address it at some point. Maybe it would get resolved in a future update? Maybe we could select which format we could export pictures to? Or maybe WordPress would add out-of-the-box pseudo native support for it, somehow?

In the back of my mind, I kept thinking about a project that Terence Eden had linked to when he wrote about this noxious format: a WebAssembly port of libheif, a library that can decode HEIF files. "What if", I wondered, "what if I made a plug-in that added this feature to WordPress?" (or maybe someone would make such a plug-in?)

But time passed and no one made the plug-in.

I was feeling very reluctant to get involved again with WordPress plug-ins to start with: I hadn't written a line of PHP in a very long time, let alone a WP plug-in. The trac page for the last plug-in I wrote tells me it's been 13 years since then.

Thirteen years.

It's basically three hundred years in technology, give or take.

Thirteen years ago PHP 5 was both THE coolest and trickiest thing in PHP land. It brought real, proper classes with it. You could do metaprogramming! We could see a future where something like "PHP on Rails" could be feasible! It fixed lots of annoying issues! But also, it was hard to find reliable support for it in servers. Lots of popular software needed to be migrated and there were lots of breaking changes that required tedious fixes which maybe were not even worth the time after all (this was before cool editors which actually parsed the code to help with refactoring script languages were widespread). And who knew when PHP 6 would be even released, was there any point in continuing this way? Why bother at all!??, etc, etc.

At the same time HTML5 was landing in the scene like a giant shiny spaceship, with JavaScript as the magnetic core that all of us who wanted to do cool stuff and SHIP IT for people to see gravitated towards... and next thing you know I am writing JavaScript full time and giving the $ signs a rest--no need to preface all variable names with a $ as in PHP, and no need to find elements with $('...'), as thanks to the then recently implemented document.querySelector we didn't even need to use jQuery for that.

So that was then. But let's stop reminiscing, and jump back to the present moment: obstacles to my blogging flow plus a very absorbing role at work translates into not posting anything in the blog for the last year.

Or in other words, I haven't told the world about the unbelievable concoctions we have come up with, and the great and terrible places we've eaten at. And as a completionist, it disappoints me more than you can imagine, as I truly enjoy reading old posts and finding what I was doing back then, and not posting for a year means a chasm has opened in the archives: The Great Disappointing Void 2022/23.

Its existence will forever haunt me, but at least it will not continue growing, as I've got over my fear of 13 years of changes in the PHP and WordPress worlds and finally wrote the plug-in I wanted: Reliably HEIC!

And now I can upload HEIC images to my blog. Ha! Success! Party! 🎉

But before I dash off to write a ton of posts to make up for the lost time, I have thoughts and reflections:

I was afraid that everything would have changed lots and I'd need to re-learn everything, but turns out that comparatively very little has changed in WordPress

I suppose this is due to the simple fact that they cannot change things substantially or they'd incur a massive backlash from their huge user base.

As a result most things seem to have either not changed at all, or slightly changed (but in a backwards compatible way), or added as an improved alternative to another existing API that was doing similar things but was not easy to change and/or extend.

Cue inevitable moments of confusion as I tried to figure out the right way these days, and realising that the documentation sometimes contradicts itself and sometimes the actual WordPress core code contradicts itself by using different styles in different places. Should I use the old or the new functions? Is anything deprecated? Can anything be deprecated here?

After spending almost a whole day trying to understand newer APIs that looked like they would do the thing I wanted, but turned out to not quite behave in the way that the cryptic documentation and the names of the API led me to understand, I gave up on trying to write "modern WP code", and instead looked at the code I wrote 13 years ago to find a working example of "how to persist a user setting in the database". Which took me 5 minutes. Thank you, past me! I saved myself from feeling like the whole day had been wasted.

At the end, I concluded that the right way was whichever way makes it work as long as it's not outright unsafe or slow, and I suspect that's the implicit guiding principle in a codebase this huge and old (20 years and counting).

Which brought to mind "The year without pants", a book I recently read about the experience of its author, Scott Berkun, working at Automattic (WordPress' owner) a while ago. After describing how "remote" looked like (a potpourri of IRC, Skype, blogs, whatever), which was interesting for historical purposes, he moves to sharing his woes and worries about organising and decision making for the WordPress project in particular and Automattic in general.

It's always intriguing to learn how organisations work internally, and it was easy to draw parallels between WP's code base and stewardship and Firefox's: to make substantial progress, you sometimes need to "rock the boat".

In programming terms, it translates into that you may need big cross-functional changes which break things and touch many people's work, and you need leadership and vision to drive these to fruition. But as organisations and code bases grow, it's somehow easier, more comfortable, to stay in your little comfortable "garden" and not really challenge other people's "gardens". Granted, your garden might be very well tended and beautiful, but the other ones might not. Their weeds might even be getting into your garden! They might have pests threatening to spread to your garden! But yet most people pretend like it's all fine and secretly spray weedkiller at night, hoping it makes the annoyances go away. I think it's human behaviour --most people avoid conflict-- but if you intend for a project/organisation to stay relevant, it quite likely needs decisive change at points to keep it in shape. Or maybe not! Maybe its users like it the way it is. Some users can be very vocal. It can be scary when you have MANY vocal users!

Food for thought, but for another day!

The PHP manual is still one of the most helpful pieces of documentation I've ever seen

You can rant all you want against PHP, but I know of few programming languages that have such a handy manual.

I think it's the combination of:

• being online AND fast to load
• easy to navigate
• most functions have examples
• contributions from other users in the comments (and most pages have contributions!)

In comparison, MDN does a really great job for web technologies in general, but sometimes the contents are far-fetched and grandiose and not as matter of fact as you get with php's manual.

Sometimes you get a full blown tutorial broken into multiple chapters with introduction, learning outcomes, "see also"... when the thing you need most is a snippet of code that shows how to tie everything together.

Other times you are left pondering whether you should look at the Web API pages or whether it's the DOM section that holds the answer to your questions, and so on. I reckon being born at a browser-maker-house causes one to not be aware of the complexity of the world view they project onto others.

Similarly, the node.js manual will leave you scratching your head and longing for an example most of the times, and you'll end up opening so many tabs to search for XYZ node.js example that you'll start to consider how to build a customised browser url bar keyword search as it would be worth its weight if not in gold, at least in typing time!

For the sake of an argument, but to be on an even footing, let's compare pages that document opening files, which is an extremely common thing to do in a server:

• node.js' fs.open is a paragraph in a very long page for all the filesystem functions and has no example whatsoever
• PHP's fopen comes with a detailed explanation documenting each flag's behaviour, four examples demonstrating different options and protocols, highlights a few gotchas and features 23 comments from users.

I hope this suffices to illustrate what I mean with "lacking examples". And yes, PHP has been around for way longer than node.js has, I give you that.

But I also say: if in doubt, add more examples. Be more like the PHP manual.

I love the Developer Tools and if you don't really know how to use them, perhaps you should start. At once!

Gone are the days where trying to debug buggy code to figure out answers to your questions would just prompt more questions because the debugger would crash on you in mysterious, frustrating and entirely non-reproducible ways (I'm looking at you, dynamic duo of Angular & Firefox debugger circa 2014).

My plug-in needs to intercept when a user selects files to upload in the browser, so that it can filter out HEIC files, remove them from the upload list, and instead read them and convert them to JPG before adding them to the upload list again.

Unfortunately, I could not find documentation for this "plugin in the middle" idea anywhere.

In fact, I suspect it was never expected that anyone would even want to alter the way the WordPress admin front-end code works, given the way it is built: without exposing any event and hardly any means to configure it at initialisation time.

What the WP admin front-end code does is they create an instance of Plupload (a JavaScript based multi-file upload library) and then they attach listeners to that instance to render things like the progress bars and potential error messages that you might get.

You can configure some options related to Plupload, but they don't expose any API for modifying the very UI WordPress creates and handles, and to make things even more exciting, the version of Plupload that ships with WordPress is customised and has things removed. So the documentation you see in the Plupload website might not actually reflect what is available in reality.

HOWEVER... poking around in the Developer Tools Console I realised that the Plupload instance is actually exposed in the global window object. Which meant I could insert my code, maybe?

I still did not know how files would flow from my local filesystem to the server via the multi-file JavaScript uploader, because there are various layers of wrapped objects and classes involved, and also they were built in a prototypesque way with libraries from before the class keyword had officially made it into JavaScript, so there were even more detours before actually significant code got to execute.

This is where using the debugger was brilliant: I looked at the scripts loaded from /wp-admin and started inserting breakpoints, watching expressions and stepping through the code to see which sort of data was being passed around, so I could figure out what sort of data I would get to receive and send.

That said, there was "a bit" of added difficulty in that WordPress ships with both unminified and minified JS code, sending the minified version to the browser but not including sourcemaps. As a consequence, I could tell which functions or entry points I'd like to see executed by looking at the unminified files, but the actual code that was executed in the browser was stripped down to the bare bones, and sometimes maybe even the function that I thought would be executed was optimised out and made to be part of some other call. So I was "a bit" in the dark.

Fortunately, the "Pretty print" feature is extremely reliable these days, and it turned humongously long lines into something a bit more readable without hanging on me at any point as was the case a few years ago!

Then, to figure out where to insert the breakpoints, I was constantly using the debugger's Find in files... feature which I'm so happy to see is super fast and stable these days too.

Of course I could not search for the actual function names because they were stripped out and optimised in the minified version, but what I would search for is 'string constants' i.e. things in quotes or specific values that minifiers can't strip out or else the code breaks. They would guide me towards the area of code I was interested in! So you could say I acted a bit like a human source mapper!

Eventually I figured out what Plupload received and expected, and I remember very vividly the SQUEAL OF JOY I let out as I managed to "trick it" into uploading a hard pink square (#ff00ff forever) I had synthetically generated using the Canvas API, exported as a PNG in a Blob and eventually sent to the Plupload instance, instead of whichever random image I had dragged into the browser. My goal was a little bit closer!

What was left to do, WordPress wise, was to figure out how to update the UI. Otherwise I would end up with "ghost" progress bars for HEIC files that would never be uploaded, as the file upload that they were meant to track had been cancelled and they would be replaced by a client-side generated JPG instead.

As I mentioned earlier, there's no API to manipulate that UI, but the saving grace in this case is that the UI is not built with Shadow DOM or React or anything like that that makes the elements unavailable to the DOM. And the element IDs they generate are fairly easy to figure out given the file the user selects... which meant I could first poke at things using the Inspector and the Console, to get a grip of where elements would be attached or how they would be named, and then write code to find the superfluous progress bars with document.getElementById, and remove them with a little element.parentElement.removeChild(element) dance.

Side note: sometimes I wonder if Node shouldn't also have a method called fallOffTheTree or getOuttaHere, which does exactly the thing as above 😆

To be honest, I'm not sure what I love more: the Developer Tools, or JavaScript, or Emscripten or... everything at once

It struck me when I was building the last pieces that involved decoding the image using a C++ library compiled for the web that it is TRULY AMAZING that we can do that, or as 'mericans would say, that it is AWESOME.

Think about that: you can circumvent whatever limitations your back-end server has to process images, and just do it in your browser by inserting some code that was meant to be run on a back-end but was made runnable in the browser, and now your browser can handle a file format it couldn't until a moment ago, and not only that, but you can combine that capability with the cool stuff the browser can do these days such as outputting JPGs... and you end up with the outcome the user wanted, i.e. a JPG in their blog.

Of course, the code that does what you need has to exist, and either you know how to use Emscripten (I will admit I don't yet), or someone is kind enough to compile the library for the web so you can use it, but do not get distracted with these technicalities:

IT IS GREAT!

We've come a long, long way in those thirteen years in Browserlandia. And again, yes, the things don't make themselves: there are working groups, and committees, and discussion groups, and disagreements, and trolls, and flame-wars, and things are shipped and then unshipped, and developers get angry and shout at each other and sometimes at you, but if you half-shut your eyes and let those incidents become a blurry blob, they're just noise concealing a nice wave of positive progress for The Web.

And with that, I'm off to write my long overdue posts.

🙋🏻‍♀️