A team is evaluating an LLM-based code generation assistant. They want to measure the quality of generated code for correctness, security, and efficiency. Which evaluation framework is BEST suited for this task?
Pass@k measures the probability that any of k generated samples pass a set of unit tests, directly assessing correctness.
Why this answer
HumanEval and similar frameworks (e.g., MBPP) use unit tests to automatically assess functional correctness of generated code, which is the most objective measure for code generation tasks.