In modern LLM applications like RAG and Agents, the model is constantly fed new context. For example, in RAG, we retrieve relevant documents and stuff them into the prompt.

The issue is that this dynamically retrieved context doesn't always appear at the beginning of the input sequence. Traditional KV caching only reuses a "common prefix," so if the new information isn't at the very start, the cache hit rate plummets, and your GPU ends up recomputing the same things over and over.

CacheBlend changes the game by allowing for the reuse of pre-computed KV caches regardless of their position in the input sequence.

This makes it possible to achieve a 100% KV Cache hit rate in applications like RAG. The performance gains are significant:

• Faster Time-To-First-Token (TTFT): Get your initial response much quicker.
• More Throughput: Serve significantly more users with the same hardware.
• Almost lossless Output Quality: All of this is achieved with little degradation in the model's generation quality.

CacheBlend works by intelligently handling the two main challenges of reusing non-prefix caches:

• Positional Encoding Update: It efficiently updates positional encodings to ensure the model always knows the correct position of each token, even when we're stitching together cached and new data.
• Selective Attention Recalculation: Instead of recomputing everything, it strategically recalculates only the minimal cross-attention needed between the new and cached chunks to maintain perfect generation quality.

An interactive CacheBlend demo is available at: https://github.com/LMCache/LMCache-Examples/tree/main/demo-rag-blending

Forget chatbots. Zuckerberg’s vision is much grander. He is betting that within a few years, AI will not just be answering your questions or writing your emails. It will be managing your schedule, anticipating your needs, running your home, helping you make decisions, and maybe even guiding your career. Call it Life-as-a-Service, powered by Meta.

The move is seen as a direct challenge to competitors. “The launch of Meta Superintelligence labs isn’t just an announcement; it’s a statement: Meta won’t settle for second place in AI,” commented Alon Yamin, cofounder and CEO of the AI detection platform Copyleaks. He added, “Meta and Mark clearly see this as a make or break moment for AI leadership.”

Catfishing just got harder in California.

A supposed band called The Velvet Sundown has released two albums of AI slop this month.

Exclusive: Shabana Mahmood told companies she wanted ‘deeper collaboration’ to tackle prisons crisis

The suited line judges have been replaced by robots. Some argue it takes the theatrics out of the court and some players have reported difficulties with the tech.

“I still want to do it.”

Archive: https://archive.is/2025.06.30-163102/https://www.theatlantic.com/ideas/archive/2025/06/customer-service-sludge/683340/

Let's Encrypt has announced it will no longer notify users about imminent certificate expirations via email due to high costs, privacy concerns, and unnecessary complexities.

The update will install much faster than 24H2.