> I suppose their utility is probably inversely proportionate to my own skill level in a given area or base competency in solving a particular problem.

Yes that's it basically.

When we solve a problem we intuitively try out strategies from solving other similar problems.

Without having much prior exposure to a special field LLMs can give you a start by finding or mixing known solutions they where trained on, but only if these solutions already exist.

They are basically sophisticated search engines with a better language interface.

Here is an anecdote where OpenAI embarrassed themselves.

It's highlighting that LLMs are not good at reasoning (thinking) but remembering (reproducing).

So as long as you ask things which require to reproduce learned knowledge they have a benefit. But asking them to solve new problems, not so much.

from https://techcrunch.com/2025/10/19/openais-embarrassing-math/ (emphasize added)

OpenAI’s ‘embarrassing’ math “Hoisted by their own GPTards.”

That’s how Meta’s chief AI scientist Yann LeCun described the blowback after OpenAI researchers did a victory lap over GPT-5’s supposed math breakthroughs.

Google DeepMind CEO Demis Hassabis added, “This is embarrassing.”

The Decoder reports that in a since-deleted tweet, OpenAI VP Kevin Weil declared that “GPT-5 found solutions to 10 (!) previously unsolved Erdős problems and made progress on 11 others.” (“Erdős problems” are famous conjectures posed by mathematician Paul Erdős.)

However, mathematician Thomas Bloom, who maintains the Erdos Problems website, said Weil’s post was “a dramatic misrepresentation” — while these problems were indeed listed as “open” on Bloom’s website, he said that only means “I personally am unaware of a paper which solves it.”

In other words, it’s not accurate to claim GPT-5 was able to solve previously unsolved problems. Instead, Bloom wrote, “GPT-5 found references, which solved these problems, that I personally was unaware of.”

Sebastien Bubeck, an OpenAI researcher who’d also been touting GPT-5’s accomplishments, then acknowledged that “only solutions in the literature were found,” but he suggested this remains a real accomplishment: “I know how hard it is to search the literature.”

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.