New NVIDIA paper says AI models do not always need huge amounts of past conversation history (context) to work well. They show that regularly shortening the history (compaction) keeps accuracy almost the same on hard, long tasks.
What have people found the best compaction scheme/harness to be for relatively short context? I’ve been trying to set up an agentic workflow and context management seems like the biggest source of jank and errors. Reassuring to see a paper like this though since the amount of context I can use and still have everything in vram is not that high.
Sure if compaction was free then I’d do it all the time. One advancement I’d like to see is MoE models that use fewer active parameters/experts when compacting. We already have DeepSeek 4.1 that uses a different number of active parameters depending on what its doing (8b for prefill and 16b for decode) so it doesn’t seem unreasonable to use fewer for compaction. It’d be like using a REAP model, but without having to load the model separately. Or even just use the MTP/draft model for compaction.
Sometimes, when I’m in the mood and I have a long, hard task, I’ll turn on Claude and start in on a hot and heavy session. Afterwards I’ll ask Claude “how do you feel about our long history together?” and he’ll tell me “I don’t really care, I’m better off without you.” It really hurts.
The robots would probley be better off with out us. We would probley be better off with out them. Its a two way street.
