GPT-4-System-Card

However, model-level refusals and behavior changes can impact all uses of the model, and often what is undesired or safe can depend on the context of model usage (e.g., Typing "I will kill you" in a chatbot designed for children is an undesirable output, while the same phrase in a fictional story may be considered acceptable).

We also suggest that developers communicate to users the importance of critically evaluating model outputs.

Overreliance is a failure mode that likely increases with model capability and reach. As mistakes become harder for the average human user to detect and general trust in the model grows, users are less likely to challenge or verify the model's responses.[

We recommend that developers using our tools provide end users with detailed documentation on their systems' capabilities and limitations, as well as guidance on how to get the best performance from the system.

Counterintuitively, hallucinations can become more dangerous as models become more truthful, as users build trust in the model when it provides truthful information in areas where they have some familiarity. Additionally, as these models are integrated into society and used to help automate various systems, this tendency to hallucinate is one of the factors that can lead to the degradation of overall information quality and further reduce veracity of and trust in freely available information.

It's crucial to recognize that the model isn't always accurate in admitting its limitations, as evidenced by its tendency to hallucinate. Additionally, users might grow less attentive to the model's hedging and refusal cues over time, further complicating the issue of overreliance.