AI

A chatbot helped more people access mental-health services

The community remains puzzled about whether these models genuinely generalize to unseen tasks, or seemingly succeed by memorizing the training data. This paper makes important strides in addressing this question. It constructs a suite of carefully designed counterfactual evaluations, providing fresh insights into the capabilities of... See more

For example, if you ask a model to “return all active users in the last 7 days” it might hallucinate a `is_active` column, join to an `activity` table that doesn’t exist, or potentially get the wrong date (especially in leap years!).

We previously talked to Shreya Rajpal at Guardrails AI, which also supports Text2SQL enforcement. Their approach was... See more

Subsequent research by Gerstgrasser and colleagues at Stanford and MIT (arXiv:2404.01413) provides important qualification: model collapse is not inevitable if synthetic data accumulates alongside real data rather than replacing it. The pathological scenario assumes complete substitution—thoughtful data curation can prevent the worst outcomes.

Observing these patterns of influence gives clues about how our models generalize from their training data. For instance, if the models responded to user prompts by splicing together sequences from the training set, then we’d expect the influential sequences for a given model response to include expressions of near-identical thoughts. Conversely,... See more

Most AI researchers roll their eyes if you bring up the idea of consciousness. That’s for philosophers, not engineers, they say. Since no one has been able to define it, what’s the point in talking about it? I get this frustration. Few concepts are as elusive and seemingly circular as the idea of a subjective experience. Despite the definitional... See more