Category: LLMs

Leveraging Constrained Sampling for Fill-In-the-Middle Code Completion

Post author By nielstron
Post date August 14, 2024
Categories In LLMs
No Comments on Leveraging Constrained Sampling for Fill-In-the-Middle Code Completion

Existing constrained sampling methods only focus on generating code that will be a syntactically well-formed program on its own after completion. What if we have some remaining code suffix and want the entirety to become a valid program? In this post I explore how to leverage existing tooling to create well-formed Copilot-like Fill-In-the-Middle (FIM) code […]

Proper Well-Formedness Guarantees for Finite LLM Sampling

Post author By nielstron
Post date August 13, 2024
Categories In LLMs
No Comments on Proper Well-Formedness Guarantees for Finite LLM Sampling

Even under constrained sampling, LLMs can produce syntactically invalid output. In this post I explore how this can happen and how to resolve the issue using results from language theory.