Insights topic
AI Systems & Agents
How modern AI systems are actually built: agent architectures, model capabilities and limits, evaluation, retrieval, and the engineering that separates a demo from something that survives production.
AI Agent Time Horizon Measurement Is Not a Uniform Capability Scale
Human task duration remains a useful anchor for agent capability, but a new audit finds that equal gains in time horizon need not represent equal gains in difficulty.
AI generated training data needs weighting, not a blanket ban
AI-generated web text can improve a data-starved model, then become harmful as stronger human corpora grow. Separate provenance and validation slices before deciding what to filter, reweight or retain.
AI agent consistency testing needs repeated runs
Average pass rates hide tasks that work once and fail on an identical rerun. Measure Pass^k, action disagreement and resulting state before calling an agent reliable.
Agent context compaction needs validated state, not free-form summaries
Agent context compaction can preserve false claims and carry unauthorised instructions into later turns. Typed state, provenance and independent validation make those transitions auditable.
AI agent sandbox security must include package infrastructure
Blocking the browser is not containment. Package registries, build workers and shared artefact stores can still provide agents with communication, code execution and indirect internet access.
Enterprise AI agent evaluation starts where APIs meet documents
Enterprise agents fail less at isolated retrieval or function calling than at the join between policy text, records and executable systems. Evaluation has to score the whole path, not the prettiest step.
Adaptive reasoning LLMs turn token budgets into routing problems
Fixed reasoning budgets waste tokens on easy prompts and starve hard ones. Adaptive reasoning LLMs shift the problem from “how long can the model think?” to “when is extra thinking worth the cost?”
AI scientific agents need evidence chains, not workflow coverage
AI scientific agents become auditable when claims are tied to evidence as they are produced. Chain-of-Evidence, omni-modal access and tool traces help, but only if the audit path is designed into the system.