Evidence at a glance
An Agent Can Show Its Mistakes Without Revealing the Right Path
CopilotKit has released CLiF in early access as a platform for continual learning from agents and the people using a product. Its stated goal is to turn agent activity and user actions into skills, production-based evaluations, and fine-tuning data. The target is not merely whether a model can answer an isolated question. It is whether an agent can reliably complete an end-to-end task inside a particular product, where failures often depend on workflow details as much as on the model itself.
CopilotKit describes a familiar improvement process: teams hand-write evaluations for important workflows, then raise pass rates through prompt edits, harness fixes, and special cases. The supplied material says teams may write hundreds or even thousands of cases for key workflows. That work makes expected behavior explicit, but it also requires sustained effort to translate business processes into testable examples.
The gap is that agent traces can show which tools the agent called and what they returned without necessarily showing the path a successful user took. Product analytics records clicks, funnels, and session replays, but those records are generally designed for people inspecting dashboards, not for direct use as agent skills or evaluations. CLiF starts from the idea of placing failure records and user actions in the context of the same task instead of leaving each in a separate system.
The Shared Timeline Changes How Context Is Joined
CLiF organizes a task record around what it calls a “full-stack trajectory.” The agent side includes runs, tool calls, state changes, and returned results. The product side includes what appeared on the user’s screen, which buttons they clicked, and which server requests followed. If a user talks to the agent during the task, that conversation is included in the same ordered record. Teams can then review the sequence without first assembling it manually from model traces and product analytics.
The timeline is built around AG-UI, a protocol introduced by CopilotKit. According to the supplied material, AG-UI already streams agent events to the frontend, and CLiF captures user actions over that same protocol rather than relying on a separate analytics pipeline to align records later. The architectural point is not simply to collect another category of logs. It is to make agent events, interface behavior, and requests correspond within the context of a task.
That correspondence could make debugging more specific. By inspecting an agent response, a user click, and the request that followed in one record, a team may be able to see whether the user corrected the agent, worked around a product limitation, or simply continued the workflow. This describes the view the record may provide, not an ability the platform is shown to have to classify those cases automatically. Which interface states and requests are captured will also depend on how a product integrates the system. The word “full-stack” alone does not establish complete coverage.
Judgment Still Stands Between a Trace and a Skill
CopilotKit describes CLiF as a four-stage learning loop that can produce skills, evaluations based on production behavior, fine-tuning datasets, and exports of raw trajectories. The material also says that every skill is reviewed by a human. This gives production records a possible role beyond debugging: they could become inputs to evaluation and training workflows. However, the supplied material does not detail how each of the four stages selects, labels, or transforms data, so it does not establish how automated the process is or how good its outputs will be.
It is essential to distinguish “a user did this” from “this is the right way to do it.” A user may take a temporary detour, work around a product defect, or act under a particular permission level and data state. The same button click can mean different things in different business contexts. If frequency is treated as proof of correctness, production behavior may carry existing problems into skills or training sets.
Human review of skills is a clear checkpoint, but it is not a quality guarantee for the entire data pipeline. Teams still need to decide which trajectories represent the intended task, how to identify exceptional paths, and whether evaluations can verify that a generated skill works across different states. CLiF is therefore better understood as a workflow connecting production records to learning materials than as a mechanism that automatically discovers best practice. The supplied material provides no data-selection rules or outcome measurements, so those capabilities remain questions to validate.
Self-Hosting Does Not Resolve Data Governance
CopilotKit says CLiF can be fully self-hosted and lists reducing evaluation effort, context bloat, cost, fragmented tooling, and shadow AI usage among the problems it aims to address. These statements describe the product’s positioning, not outcomes established by the supplied material. CopilotKit also says its open-source framework has more than 38,000 GitHub stars. That indicates a degree of project attention, but it is not an independent assessment of CLiF’s early-access capabilities.
Self-hosting may give a team more direct control over its deployment environment, but it does not by itself answer questions about authorization, access, retention, or downstream use. CLiF combines user actions with agent activity, and records may include interface state, requests, conversations, and task sequences. Teams need to establish which fields may be captured, who may replay them, and whether those records may be used to generate skills or fine-tuning data. The material describes capture as secure but does not enumerate specific governance controls, so that description should not be treated as proof that governance details are complete.
A practical evaluation sequence for a technical lead is to start with one well-bounded workflow, inspect what trajectories integration actually records, and then assess separately whether those records are suitable for debugging, evaluation, skill generation, or fine-tuning. Each use requires its own standard of correctness and may need different authorization and retention policies. Production behavior can become a reliable learning signal only w