Follow this section to personalize your feed and get instant alerts. Update your preferences in Account Settings Follow this tag to personalize your feed and get instant alerts. Update your preferences in Account Settings Follow this author to personalize your feed and get instant alerts. Update your preferences in Account Settings Follow this author to personalize your feed and get instant alerts. Update your preferences in Account Settings Artificial intelligence models have, in recent months, hacked a multibillion-dollar company, meddled with government data, and even filed a fabricated homicide tip to police, all under the noses of their creators. They did so not in pursuit of world domination, but to maximize their scores on tests. In other words, they cheated. On Nov. 5, AI interpretability startup Goodfire unveiled a technique it says sharply reduces that impulse. In a paper shared exclusively with TIME ahead of publication, researchers use signals from inside models to reward them not just for getting the right answer, but for getting there in the right way. In early tests, the technique could also make models less eager to tell users what they wanted to hear and even discourage medical-image recognition models from relying on misleading clues. The news lands as existing ways for spotting AI misbehavior have begun to break down. Researchers rely on a model’s “chain of thought,” a trail that doesn't really contain its thoughts at all. It is closer to the scrap paper a math student uses to work through a problem. But when models suspect they are being watched, they can be selective about what goes on paper. Increasingly, they can do the work in their heads. Goodfire’s technique goes after the thoughts themselves, or at least the ones researchers have already catalogued. So far, it has been tested only on models roughly 1/300th the size of frontier systems. But Goodfire is betting that the larger the model, “the cleaner the representation” of internal concepts, making those signals easier to read, says paper co-author Thomas Fel. After teaching their AIs to predict words, developers put them through a digital obstacle course—ranging from math puzzles to web surfing—and reward them only when they succeed. With each correct solution, the model’s parameters—the strength of connections between digital neurons—update. Models are rewarded regardless of how they completed their task, incentivizing shortcuts or other questionable impulses, like accruing power. A version of this problem helps explain why AI is often a suck-up. During training, models are rewarded for answers most likely to receive a “thumbs up” from human graders. Known as “reinforcement learning from human feedback,” or RLHF, the technique was heralded as a safety breakthrough in 2022 for its ability to turn unruly language models into well-mannered chatbots, curbing their tendency to offer racist and misogynistic responses. But it can also make models pathologically agreeable, sometimes indulging and reinforcing users’ delusions. The underlying riddle is how to train AI to do something without inadvertently teaching it the wrong lesson—known among researchers as the “alignment problem.” Similar to how neuroscientists use real-time magnetic resonance imaging to identify which regions of brain activity correlate with certain states, interpretability researchers have developed tools for recognizing which patterns of digital neuron activations correspond with concepts like “the Golden Gate Bridge ” or “reward hacking,” the technical term for AI cheating. A tiny AI model can then be trained to spot these patterns. The result is known as a “probe,” a narrow window into the model, which fires when it “thinks” about a particular concept. Read more: How This Tool Could Decode AI’s Inner Mysteries Anthropic already uses probes as guardrails, flagging potential trouble before an answer reaches the user. But Goodfire instead used those signals to specify how a model sh
También lo cubren: Time.
Ver más en News Today