Most of my initial use of ChatGPT was of the gee-whiz, what else can I make it do sort of poking and prodding. Make me a children's story for my kids. Tell me everything about Spinosaurus. What do you think of QuantumScape ($QS) stock? I received thoughtful answers and was duly amazed.
When it came time to try to do some of these tricks at work, I hit a wall. Telling the LLM to 'analyze this data and find trends of interest' was turning up pretty poor results. The classic 'make this document more professional/smart/friendly/etc.' turned out documents that had a familiar LLM air I couldn't quite put my finger on. Yes the hyphen usage was a tell but so were the overlong sentences, the repetitive beating a dead horse approach to proving a point, using pretty cliched points of humor that were a bit off for what I was writing.
The man in the mirror
As is often the case, my problem was me. My prompts were too simplistic, so the LLM was trying to solve a general problem as best it could. The first step on my journey was prompting better. I was naive enough to ask ChatGPT and Claude how I could prompt better and got amazing results. I wanted to make a silly image with Grok and was getting nowhere until I asked it how I should make the request. Weirdly, when responding to that inquiry, it seemed to know exactly what I was trying to do, whereas submitting the inquiry itself led me down a blind alley.
The advice
- Assign a role. You are Jack's Lung (for my fellow Chuck Palahniuk fans). You are a world class UX designer. Take on the role of a Wall Street analyst. You are a technical writer specializing in electric vehicle charging protocols. Fantasize about who you want to help you. I once told ChatGPT to make me a recipe and pretend you are Anthony Bourdain. I got back a hilarious, PG-13 rated recipe that had political commentary and a purist streak.
- Describe the task in detail. Create code that does x. Write me a 1000 word blog post. Create a spreadsheet comparing strengths and weaknesses of these competitors [x,y,z].
- Define the output. Save the file in markdown in this directory. Save this as a CSV. Provide me with a list of code changes. When I approve the changes, then write the code.
- Define the audience and context. This is for me, a demanding product manager, so I can write the blog post and pass it off as mine (what is authorship these days?). This is for a technical audience of programmers. This is a bedtime story for my young boys.
- If it is writing as the output, talk about what the style, tone, and language should be. This should be professional. Sentences should be as short as possible. Avoid jargon. Use industry jargon.
- For content, it is very helpful to give discrete goals. Provide three reasons why Charmin is the best TP. Provide counter-arguments to this academic paper's main arguments. Explore where this hypothesis will fail. If the output is a spreadsheet, define the columns and rows.
- I almost always tell AI to take your time and don't spare my feelings. I find just allowing for the patience almost always yields better results as the model is not optimizing for speed. The cloying compliments are distracting when I have a work task; I need more critique to be successful.
- The medium matters, as Marshal McLuhan surely knew. A post for LinkedIn is not the same as an article in the New England Journal of Medicine. This will be submitted to a popular magazine. This is going to be part of my product documentation.
Prompt like a pro
This great e-book had a couple of great frameworks. One is RTCAC: Role Task Context Output Constraints. You are a professor specializing in bugs. Write a 4 paragraph description of the bug in the attached photo describing mating habits, diet, where it lives, interesing trivia, distinguishing traits, and general information about this bug. Use tone and language suitable for my 4 year old.
This work also adds that it is helpful to be specific aobut the role: You are an expert in bird law. You are a senior software architect for SaaS platforms. I have stacked competencies with my AI agents as this text recommends. You are a world class UX designer and technical writer that specializes in EV charging technology.
I don't do this enough, but it is recommended to do 'chain of thought & step-by-step thinking' with AI. Break the task into steps ahead of time, and either tell the LLM to explicitly "think step by step" or for each step, make that its own discrete prompt.
Overview
I did best with LLMs and agents when I took on the role of a manager with a new, young hire. Provide confident, explicit instructions that didn't meander. Provide continuous feedback on what you like and don't like. Refer back to some foundational principles for the work. In the example of some apps I have created, I picked a design style "maximalist" "minimalist" and made sure to keep referencing that when making UI changes.
While Claude's agents could seemingly take on a very long set of tasks with ease, I found that it also meant more extensive QA work than tasking the agent with smaller chunks of work. It was easier to get over your skis and find that game of AI whack-a-mole happening where a bug fix leads to some other problem popping up, and the fix for that problem leads to the original bug coming back or two new bugs and you are 12 changes deep before you realized some page completely disappeared and you rm -rf the-project and being anew.
Have your new best friend AI model create a first draft with the RTCAC method. Provide specific, actionable feedback. Or, give it vague feedback to spin the random button "Make this more professional" and see a totally difference spin. Keep iterating and treating the LLM like a friendly direct report looking for feedback.