Why asking someone technical for their prompts results in...

You've possibly asked someone yourself. Thinking "maybe I can get the same quality as that person if I just have the right AI prompt".

If you're a technical or specialized person you probably can relate to this. You've been asked to share the prompt you used to do something, and you get this feeling of "ugh". It's not the actual task of sharing the prompt that is the issue.

I'm going to put into words what is hard to explain in a brief conversation so both sides of these conversations understand what's going on.

Here's the difference between the prompts each person is writing.

When you're not-specialized in that area

Let's assume you don't have much domain knowledge of this area of expertise, but you have access to a good AI tool. Your prompts will often look like this:

Make me a (high level output), no mistakes.
I want a (high level output), that has (adjectives describing appearance and maybe some functionality but without the technical terms)

Substitute in whatever high level thing you're trying to have it produce, be it a website, report, or other type of output that benefits from specialized knowledge.

We've all experienced this - trying to get AI to build something that falls outside of our own domain of knowledge and experience. Maybe you tried to look up something medical, maybe you looked up something related to cars, home improvement, or maybe you decided on a whim you wanted to take up a new hobby after seeing something cool or pretty that someone else made.

What do you typically get back?

You get something that might generally be in the direction you were wanting but is far from being the complete or perfect picture you were imagining in your head. Maybe you see someone else seemingly having more success than you with prompting. What is setting you apart? It could very well be that they have some AI skills installed, but it could be more tied to your own "skill"/ knowledge.

When you are specialized in that area

Your prompts look pretty different when you do have domain expertise in the area and you often start from a completely different place.

Your prompts often:

May set the overall goal as building a (high level output) but you are specific in the smaller details of what you want.
May accomodate for common failings of AI in your domain area, that you've experienced. For example if you build websites, you may know that it tends to default to building in React which may not be appropriate for the task you actually have, so you instruct it more specifically in what frameworks/tools you want used.
You explain something complicated in less words than it would take for someone not specialized in your domain. The reason is you know the terminology, how the task is commonly done etc.
You're giving the agent context on the problem you're solving for example
Your prompts are the output that make AI do what you want but it's not magic, you didn't find these words in a long lost ancient tome. You were just able to say what you want with specificity and explain the process you want the agent to perform.

Why someone who is specialized gets that feeling of "ugh" when asked for their prompts

If you're not specialized in that area you may feel like they're just being selfish or what's the big deal it's text? It's not that.

That feeling for the specialized person comes from knowing:

Each prompt is specific to the thing you're trying to produce. If you want a specific outcome you provide a specific prompt not something general. You describe how you go from nothing to something, the actual process you want followed.
The person asking, can't understand the terminology or underlying concepts and relationships used in the prompt and therefore is not going to get the thing they actually want.
The prompt was written quickly, you wrote it knowing you probably weren't going to 1-shot prompt your way to the result, you just wanted to start and iterate. This means it's either not 1 prompt or it's a prompt that you know is just a slot machine that may sometimes produce a good result sometimes a terrible one. The person asking you, may not be able to tell the difference between the good and bad.
If the person gets a bad result or one with errors they don't know how to troubleshoot and solve those problems and will either blame the prompt from you or give up frustrated.
Explaining any of the above like the terminology, quality etc, isn't what the person asking likely wants. They want the cheat code. They want the easy answer that doesn't require learning something.
The specialized person even knows what they don't know and that enables them to accomodate for their own lack of knowledge on something niche.

AI skills

AI skills are different. When built well they are modular, they describe how each tiny piece of the bigger picture is to be thought about. They may not state the end result, but describe the desired processes needed. When describing an end result they state the way you typically want things of that nature created in a reusable way. These skills can be mixed and matched across different tasks and possibly even domains.

This adds another layer to the "ugh" factor when asked for your prompts:

The specialized person has created or installed AI skills specific to their area of expertise. They knew which skills were actually going to be useful and which were not. The agent isn't just seeing their prompt it's using these skills. So what the specialized person gets out and what you get out are not going to be the same thing.

You might then think, I should just install a whole bunch of skills related to that domain and then prompt:

While it could work, odds are likely you're going to just increase your token bill because you're not intelligently picking the set of skills you want the agent to use, you're throwing tons of possibly unrelated text at it and hoping it finds it's own way.
You may even overload the model's context window making it less useful. All LLM models have a needle in a haystack problem, the more information you give them after a certain point, the harder it is for them to identify the correct thing.
You're also likely giving the AI conflicting skills (instructions) that it has to try to reason about. In software development different engineers have different levels of experience, different industries they work in, and also may prefer certain ways of doing something because it's better for their specific role or fits their existing knowledge better. So the authors of these skills can author them completely differently and you're forcing the AI to try to reason about them.

Needle in a haystack is a test where LLMs are given a massive amount of text and told to find something specific in it. After a threshold of context window usage the LLMs tend to start performing worse.

How do you write better prompts?

You're not going to like my answer. You need to learn more about the domain of expertise you're trying to leverage. There are ways of kind of trying to cheat it like having AI perform research, then construct a plan, then execute the task, but when you don't get the result you want then you can't steer the ship back to what you want. You also may get in a situation where you don't know what the AI produced is bad, and you trust and rely on it and it harms you.

You don't need to become a grandmaster certified in everything related to that domain. Any small knowledge gain in that area improves the results you'll see.

How do you learn that domain of expertise?

You might have guessed it but, it's gotta be the old fashioned way, Googling, trying and failing, asking experts questions for understanding, not for prompts. Take online courses, watch videos on that domain (not on prompting), befriend folks who are in that field, talk to them about what they do.

This is not a "Don't use AI to learn", old man loudly shouting "back in my day..." statement.

Asking AI can help but there's a fundamental problem. If you don't learn the foundations of something you can learn a foundational piece incorrectly that then acts as a poisoned apple to the rest of your learning. AI still hallucinates, AI still gets things wrong. Ask any specialized person about a time that AI did something dumb in their area of expertise and they'll have more than one usually funny story. So if even the experts in the field are getting bad results from AI sometimes, of course you who doesn't have the knowledge will also get bad answers, and what you need to do is insulate your learning against the bad AI information so you can help identify those.

Not only that, if you consistently use AI to solve problems for you, you don't learn the lessons. The next time you hit a similar problem, you'll reach for the AI tool again and be rolling the dice hoping you get a good result again. Not only that once you are in a job in that field you become just an added person in a game of telephone between the customer's wants and the AI. The company doesn't need layers of prompt delegation. They need someone who can think critically and discern good versus bad results, chart the path to success, fully understand the customer's problem on a deeper level.

No AI was used to write or edit this article. Gen. AI used to make the cover image based on real photography.