I'm also between unimpressed and useful.

Usually, I have no need to ask an AI for code. I may use it as a better search engine, so I don't have to spend as much time reading the documentation to find the one bit of information I need at any moment. But today I had a coding task I wasn't familiar with.

The first step was to add something to a found Python script that used the fontforge API to render out glyphs. I'm not fluent in Python (aside from the general syntax, but it's the libraries where the real magic lies), so I asked Gemini. The result was correct, it worked perfectly (including the mistake I made in my prompt), it looked well-structured and documented with everything a coding noob would want to know about it.

With that positive vibe, I thought I might give Gemini a run for its money. I had planned on post-processing the images using two batch-processing tools, which would have taken me five or ten minutes, tops. But why not see what the AI can do?

Perl 5.42: I need a script that goes over all PNG files in a folder and converts the image by: (a) changing the canvas size to 1024x1024, centering the image if it is smaller. (b) replacing the background color of #FFFFFF with #6A4B31 (which also applies to the new regions added in step a). (c) Replaces the foreground color of #000000 with #E0EA3E. (d) saves the file into a new subfolder "processed" with the original filename.

Now, that is something I can judge and judge harshly on. While Perl may not be my strongest language anymore---earning money with Java for 25 years pushed Perl down to second place---I know it well. What I did not know at all was image processing in Perl. I assumed it would pull in something like Image::Magick, but I was vague on knowing what other "somethings" are out there.

The result of this initial query was impressive. I got a 102-line script, using use strict, warnings, Image::Magick, and File::Spec, and doing so correctly. It was nicely laid out, with definitions up top and comments all over, e.g.:

# (b) Background color replacement
my $color_bg_old = '#FFFFFF'; # White
my $color_bg_new = '#6A4B31'; # Brown

# (c) Foreground color replacement
my $color_fg_old = '#000000'; # Black
my $color_fg_new = '#E0EA3E'; # Yellow-green
[download]

Ok, there was no "(a)", but it did even identify the colours and name them. There was error handling, and even a statistics print at the end about processed and errored files. And the script seemed to be exactly what I asked it to, and was coded in a way I could not fault. Sure, "$err = ...; if ($err) {" is not how I would have done it, but it's not wrong per se.

So far, I was impressed. Until I ran it and it blew up.

Someone knowing Image::Magick might have spotted it, but I do not. It turns out, Gemini is really bad at selecting the correct parameter names, or even getting the method name right regarding uppercase and lowercase letters.

Side tangent: Grammarly, an AI-powered spell checker and grammar checker (and more nonsense I have turned off), is trying to get me to replace "really bad" in the previous sentence with "evil". What in the world? The tool that's annoyingly always trying to take out anything that's not soft as a pillow wants to throw in "evil"? It must have a strong opinion on Gemini...

Anyway, this time Gemini could fix the mistake after I pasted the error message in. But then I noticed that I had misjudged the files. They weren't black and white; they were greyscale. So I asked Gemini to update the script to colour-change greyscale to a blend of the new colours. And, boy, oh, boy, this is where the fun began. The next 20 minutes were filled with Gemini trying harder and harder to do the job while messing up parameter and method names, and when the script ran but produced nonsense, it alternated between blaming my Image::Magick being too old (I just installed it, it's as new as it can get) and blaming the files for being encoded weirdly.

Yes, I was stubborn, I admit that. I could have taken over debugging and finding out the right calls way earlier. But I had fun seeing the AI struggle harder and harder. But even my patience has its end, and at some point, I wanted a working script. So I looked at it closely and noticed two things: (a) "$image->CLUT($clut)" looks suspiciously different from the other calls and (b) that $clut image it produced for that was 1x1 pixels in size. (That was from the initial reply; I ignored the ten iterations where it added and changed all kinds of stuff.)

With my diagnosis, it then managed to fix the script. The last mistake (it switched to using the wrong background colour, which was correct for the black and white version) I fixed myself.

All in all, I have mixed feelings. On one hand, it saved me an hour or two looking into Image::Magick and finding out how to do the conversion. Something I would probably not have done for a one-off script, as I had another toolchain ready. But on the other hand, it couldn't get the script into a working state without some trivial (method name) and some not quite as trivial (1x1) human intervention. Then there is the rest of the script. Yes, it did write that for me, but to be honest, looping over files, creating a directory, printing some feedback, and so on, I could have written as fast as I can type. There wasn't really any savings on time or effort there. Then, what did I gain from this exercise? I could have asked it to grep me the relevant parts of the Image::Magick documentation, i.e. how to read and write a png, and how to swap colours. Compared to what I want through, that would have been faster, assuming it directed me to the right parts of the docs.

I can't say I'm unimpressed, but I also can't call it really useful as a coding assistant. It's a great research assistant. That's what I've been using it for quite a while now, and that's what I intend on doing. Will I ask it for ready-to-use code again? Maybe. Getting the structured file was nice, even if I could have written that myself from memory. So even if the core of the solution turns out worthless, there's at least that.

But where I see a big issue is in skill. I built up the skill I needed to get this running over many years by reading documentation and trying stuff out myself. What will happen to the next generation, especially if AI gets better at fixing its mistakes? How will they build up their skills to tackle the hard stuff if they get the simple stuff from an AI?

I even see this a bit in myself. When I started using some new library or tool, I used to read the documentation top to bottom. That's a great way to really understand how something works, and you pick up what else you can do with something along the way. That is slowly falling by the wayside, as I can get a summary of just the parts I need at the moment in seconds. For example, I now know nothing more about Image::Magick but that it can read and write files, can replace colours, and can apply a LUT. Fr me, this is "I learned nothing". I have taken no personal gain or skill growth from this exercise. Sad, isn't it?