LLM: Digital interns and when to use them

Foreword: This post is partly serious and partly ironic.

At first we witnessed the rhetoric of AI that – it was said – would kill all jobs.
Now we are at a more reasonable consensus that recognizes that jobs are composed of many tasks and that AI automates tasks, not jobs.

Then we saw a rhetoric that generative AI would replace humans in all creative activities.
Now it seems to me that we are coming to a more reasonable consensus that recognizes that the very feature of algorithms that gives systems plasticity to adapt to new situations is the cause of inaccuracies in ouputs.

You can’t take the output of an LLM for gold, it has to be checked to avoid errors and to align the output with our desiderata, but they are a good tool to produce a semi-finished product for refinement.
Somewhat like work done by an intern: it produces semi-finished products to be checked and finished.

Rather than taking work away from professionals, they will increasingly be joined by digital interns – the LLMs.
It may well be the interns who will suffer most competition from LLMs….

But does it make sense ?

For a while now for every task I’m working on I ask myself: would it make sense to delegate this task to an intern and just refine the output ?
The answer is generally negative.

Delegating requires time and instructional costs.

Controlling the output is necessary.
I am responsible for my own output.

For it to make sense to delegate, the time invested in instructing must be justified by the
– duration of the task
– time of my correction
multiplied by the number of occurrences of the task, and adding the time and setup cost, must be significantly less than the time I would spend if I did it myself.

Wanting to sketch out a formula… I delegate to a digital intern/LLM if the human execution cost of the number of tasks is >> to the cost of execution with an AI of the same number of tasks

The human execution cost is composed of the task execution time th(t) times the unit cost of human time ch(t) times the number n of tasks performed

The realization cost with an AI is composed of setup costs and execution costs.
Setup costs are related to time spent by humans th(s) times their cost ch(s) plus technical setup costs ct(s)
Execution costs are composed of human correction costs and unit costs of AI use cu(ai) , all times the number n of tasks executed
Human correction costs are composed of the human correction time th(ct) times its unit cost ch(c)

In the end, we can delegate to an AI if

th(t)*ch(t) *n >> th(s) * ch(s) + ct(s) + ( th(c) * ch(c) + cu(ai) ) * n

As I said, I do not find that this can be often true in my normal activities

An example:

I have a speech-to-text converter on my pc.
it is much faster at writing than I am but I don’t use it because the review and correction time exceeds the benefit in transcription since and the type of output is different (the sentences we produce when we speak are different from those we produce when we write), so I have to correct a lot and the benefit is nullified

Perhaps a parameter of psychological “frustration bias” should also be included : maybe overall I would be better off using text to speech but those times when I have to correct a lot deter me from using it more frequently because of a sort of “I do it, I do it better, I do it faster” effect

The frustration bias b(f) should correct the bias that causes us to use the digital intern/LLM less than we could: the lower the frustration bias the greater the convenience of using the tool

The final formula would then be….

th(t)*ch(t) *n >> th(s) * ch(s) + ct(s) + ( th(c) * ch(c) * b(f) + cu(ai) ) * n

If you like this post, please consider sharing it.

Leave a Comment

Your email address will not be published. Required fields are marked *