How to review generative AI outputs

Why gen AI content requires review

Gen AI tools like Copilot Chat can help you save time and get started on tasks. But the information these tools provide isn’t always accurate, complete or appropriate to share.

AI can analyze information, suggest ideas or highlight patterns, but it cannot:

• Understand people’s lived experiences, values or cultural context
• Fully understand your specific role or workplace without clear direction
• Decide what information is appropriate
• Guarantee that its outputs are always original or correct
• Understand why something is unclear or inappropriate

Check if the response makes sense

Begin by asking yourself if the response fits your request. You can do this by checking if the output:

• Is on-topic and coherent
• Answers your question(s) in full
• Provides the right level of detail
• Uses a tone that fits your needs
• Is consistent, especially in longer responses

If the output is flawed, try rewording your prompt.  

Review for accuracy

Gen AI tools are trained on very large datasets that include text, images, and other types of information. They learn patterns from this data to generate new content, rather than understanding the information.

AI tools can produce results that sound convincing and well-written but are factually wrong. They can generate plausible-sounding responses by guessing what “could” be true, even if it’s not. This is called “hallucination” and happens because AI tools predict words based on patterns in their training data, not on real understanding or facts.

Because of this, it’s important to always verify information is true and fact-check details like:

• Names of people, places and organizations
• Dates, numbers and statistics
• References to laws, policies and procedures
• Quotes and citations

This can be done through primary and authoritative sources, like official government or organizational websites, original documents or data and peer-reviewed publications.

Some outputs may claim to be based on legitimate sources and include links. But these links can be outdated, lead nowhere or point to unrelated material. This is a common way people are misled by AI-generated content. Always check that sources are real, relevant and reliable.

AI tools may also generate contradictions. It’s not uncommon for a tool to say one thing in one output and the opposite in another. Ensure you’re checking for consistency across the entire response.

Know your audience

Even when using Copilot Chat with IDIR enterprise data protection, it’s still your responsibility to check the output. If your prompt includes or hints at internal information, AI tools may reflect that in its response. For example, it can include:

• Names of real people 
• Project names or codes 
• File names or folder paths 

Read every output carefully to ensure any included personal information, confidential data or sensitive details are appropriate for your internal or external audience.

Watch for bias

AI tools do not understand fairness or context the way people do. As a result, they may unintentionally produce responses that include stereotypes, use biased language or reflect historical inequalities. This can show up in both obvious and subtle ways, including how people, groups or issues are described.  

As the human user, you are expected to be alert and check your outputs for inclusion. When reviewing an output, ask yourself:

• Are some identities portrayed more positively than others? 
• Are any important perspectives or demographic groups missing? 
• Does this subtly favour one viewpoint over another? 
• Are certain perspectives assumed to be “normal” or default?  
• Does the wording rely on cliches, assumptions or generalizations about a group? 

If the output feels biased, do not use the content. Consult with subject matter experts for more complete or accurate information, or to help craft a more effective prompt while being mindful of bias.  

Getting another perspective can help reduce the risk of harm and ensure messaging is respectful and accurate. Your policy or communications team may be able to help assess the content’s tone and balance.