# Evaluate a prompt before and after optimization

> Use examples and feedback with OpenAI's prompt optimizer, then check whether the revision improves real outputs.

Source: https://fullbytedeveloper.com/blog/openai-prompt-optimizer-evaluation
Originally posted: 2025-09-03
Published here: 2026-11-06

OpenAI's prompt optimizer can suggest a revised prompt from examples and feedback. Its documentation also warns that an optimized prompt can perform worse on particular inputs [1].

## Prepare examples first

The documented dataset workflow starts with at least three rows containing responses. Each row needs a grader result or human annotation. Feedback can include good or bad labels, written critiques in `output_feedback`, and grader results [1].

Use examples that expose the problem you want fixed. If the failure is missing a required field, judge that field directly. A general instruction to make the answer better gives you less to compare.

## Review the proposed revision

Choose Optimize in the prompt pane, then inspect and test the resulting prompt. OpenAI recommends narrow graders for the properties that fail and manual evaluation before production use [1].

Keep your original prompt so you can compare both versions on the same examples. Add a few cases that were not used to guide the revision. Accept the change only when the outputs support it.

The original social post called the tool free. That caption should not be read as a promise that model calls or evaluation work carry no cost; check your account's current billing before running a large dataset.

## Sources

1. [OpenAI: prompt optimizer](https://developers.openai.com/api/docs/guides/prompt-optimizer)

## Original posts

- [Instagram post](https://www.instagram.com/fullbytedeveloper/p/DOHl0TBjGeX/)