I actually wonder if that’s a benefit for young people just starting out on their career journey. It’s mostly about feelings and a general sense and not specific opportunities to advance a career. In a lot of ways, a well established manager whose from another generation is not in time with those feelings and the difficulties with navigating them in a complex corporate environment.

There’s something to be said for the abilities of a tool reflecting its wielder.

In research circles, the most advanced pipelines in terms of prompting have a 90% success rate at things the same model only gets right around 30% of the time with naive zero shot prompting.

At a minimum, people should be familiar with chain of thought prompting if using the models. That one is very easy to incorporate and makes a huge difference on complex problems.

Though for anyone actually building serious pipelines for these products, the best technique I’ve seen to date was this one from DeepMind:

We introduce SELF-DISCOVER, a general framework for LLMs to self-discover the task-intrinsic reasoning structures to tackle complex reasoning problems that are challenging for typical prompting methods. Core to the framework is a self-discovery process where LLMs select multiple atomic reasoning modules such as critical thinking and step-by-step thinking, and compose them into an explicit reasoning structure for LLMs to follow during decoding. SELF-DISCOVER substantially improves GPT-4 and PaLM 2’s performance on challenging reasoning benchmarks such as BigBench-Hard, grounded agent reasoning, and MATH, by as much as 32% compared to Chain of Thought (CoT). Furthermore, SELF-DISCOVER outperforms inference-intensive methods such as CoT-Self-Consistency by more than 20%, while requiring 10-40x fewer inference compute. Finally, we show that the self-discovered reasoning structures are universally applicable across model families: from PaLM 2-L to GPT-4, and from GPT-4 to Llama2, and share commonalities with human reasoning patterns.

• https://arxiv.org/abs/2402.03620

So yes, maybe you aren’t getting a lot out of the models. But a lot of people are, and the difference between your experiences and theirs may just boil down to experience in using the tool. If I just started using Photoshop for an hour or two I might complain about how the software sucks at making good looking images. But we both know it wouldn’t be the software’s fault.

Surprisingly, I’ve had the opposite effect. Wherein, it has increased my productivity by tenfold and has helped with code review and/or confirming various logic, etc. Although, I wouldn’t necessarily take what it tells me as gospel from a recommendation standpoint in terms of my career as a whole. I’ve definitely caught it numerous times being wrong, but the inaccuracies pale in comparison to what it gets right, imo.