Mastering prompt writing for large language models

The importance of LLMs and RAG models

It's crucial to understand how LLMs and RAG models function. These models operate similarly to humans by using context to answer questions. For instance, if you were to ask a colleague about a specific tax issue, you would provide context and details to help them understand the question. LLMs work the same way but on a much larger scale, using the entirety of the internet as their context.

Writing effective prompts

To get the best results from LLMs and RAG models, follow these key principles:

• Be Human: Talk to the AI as you would to a colleague. Provide context and details to make the question clear.
• Provide Context: Include as much relevant information as possible. This helps the model understand the question better.
• Use Keywords: Incorporate keywords that are likely to be found in the answer. This helps steer the prompt in the right direction.
• Be Specific: Clearly specify what you are looking for, including details like the tax year if relevant.

Examples of good and bad prompts

Understanding the difference between good and bad prompts is essential for effective communication with AI. Here are some examples:

• Bad Prompt: "Illinois, Michigan, reciprocity agreement" – This prompt lacks context, specificity, and question words.
• Good Prompt: "I have someone looking to file a non-resident 2025 Illinois income tax return. They are a resident of Michigan. Is there a reciprocal agreement that means they don't have to file a 2025 Illinois tax return?" – This prompt provides context, specificity, and a clear question.

Follow-up prompts

After receiving an initial answer, use follow-up prompts to refine the response. You can ask additional questions to dig deeper, request primary sources to support the answer, and format the response as needed (e.g., as an email, memo, or research paper).

Interactive example