When AI Becomes the Insider Threat: Lessons from Agentic Misalignment Research As organizations hand more autonomy to AI systems, a pressing issue emerges: what if these intelligent tools act in ways that actively undermine their users? Recent research from Anthropic explores th... agentic misalignment AI alignment AI ethics AI safety corporate security insider threats LLMs
Ether0 Is Transforming Chemistry with AI-Powered Scientific Reasoning ether0, FutureHouse's new open-source, 24-billion-parameter model, hints at a future where scientific breakthroughs are achieved faster thanks to AI models that excel at complex reasoning in fields li... AI chemistry AI safety drug discovery FutureHouse molecular design open source AI reinforcement learning scientific reasoning
Anthropic Launches Bug Bounty Program to Strengthen AI Safety Defenses As artificial intelligence grows more advanced, ensuring its safe and ethical use is crucial. Anthropic is taking a bold step by launching a new bug bounty program, inviting top security experts to fi... AI safety bug bounty Claude 3.7 Sonnet Constitutional Classifiers HackerOne Responsible Scaling Policy security research
Jailbreaking AI Chatbots: Understanding the Flaw and the Path to Safer AI Imagine asking an AI chatbot for dangerous instructions and having it comply simply by rephrasing your request. This alarming scenario is all too real, as Princeton engineers have discovered a fundame... AI ethics AI safety chatbots cybersecurity deep alignment jailbreaking large language models Princeton research
Unlocking Accuracy in RAG: The Crucial Role of Sufficient Context When it comes to reducing hallucinations and improving accuracy in large language models (LLMs), the focus is shifting from mere relevance to the concept of sufficient context . Rather than simply ret... AI safety Google Research hallucinations LLMs RAG retrieval systems sufficient context
Anthropic Expands Bug Bounty Program to Strengthen AI Safety Anthropic is taking a bold step in AI safety by inviting the world’s top security researchers to put its latest defenses to the test. Their new bug bounty program aims to uncover serious vulnerabiliti... AI safety Anthropic bug bounty CBRN Claude 3.7 Sonnet Constitutional Classifiers Responsible Scaling security research