PDF.

A study from Profound of OpenAI's ChatGPT, Google AI Overviews and Perplexity shows that while ChatGPT mostly sources its information from Wikipedia, Google AI Overviews and Perplexity mostly source their information from Reddit.

• Deep in the Amazon, pharmacists are using AI to process prescriptions in overstretched public clinics.
• Developed by Brazilian nonprofit NoHarm and backed by tech giants including Google and Amazon, the AI assistant helps health-care workers catch errors.
• Early success suggests it is a scalable model for AI in under-resourced health systems.

• Following the launch of ChatGPT, freelance workers quickly changed how they work and what jobs they pursue.
• Higher-skilled freelancers began bidding on a broader value range of jobs and proportionally fewer high-value jobs as a means to stay active, maintain their visibility and reputation, and adapt as market dynamics evolve.
• AI helps lower barriers of entry to opportunities: More freelancers from outside the U.S. entered the market, using AI tools to compete in new job categories.

Using public information and making small tweaks, an alpha-seeking AI fund manager outperformed 93% of mutual fund managers by an average of 600%.

• HTML.
• PDF.

In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.