iCreatAI News
Back to main site
Skip to content
#Safety / alignment
Nov 10Tue
  1. Hacker News AI88

    AI R&D Is Accelerating Towards Potential RSI in 2027

    METR's deep-access investigation of Anthropic found that since October 2025 Anthropic's models have been improving on its internal AECI benchmark at roughly 1.9x the pre-October 2025 rate (95% CI 1.5–2.8x), with a majority of the acceleration likely attributable to AI doing R&D and no observed bottlenecks preventing further acceleration. METR reports that Anthropic redacted most of the evidence it found most informative, and critics such as Toby Ord argue the data are also consistent with a one-time level shift from the larger Mythos pretrain (implying ~1.4x). The post's author calls the situation grim and argues timelines may be shorter than the AI 2027 scenario, while noting the estimates could be wrong or bottlenecks could be hit.

    Why it matters: Summarizes a widely discussed METR report on AI-driven acceleration of AI R&D and the debate over whether it implies near-term recursive self-improvement.

  2. The Decoder58

    Tencent's AI chief says AI bioweapons are coming, but closing off models isn't the answer

    Tencent chief AI scientist Vinces Yao Shunyu argues for a mix of open and closed models, saying open models also let defenders use the technology, though he notes offensive and defensive capabilities are harder to separate in biology than in cybersecurity. The article examines the limits of his Hugging Face example, the fragility of open-model safety filters via abliteration, and Anthropic's counterargument that published weights can't be taken back. Tencent plans to release its Hy4 model as open-weight with a safety report, saying its biological capabilities fall below the high-risk threshold.

  3. The Verge · AI38

    Hold up — I had a great time talking to the RIAA guy

    In this Decoder interview, RIAA CEO and chairman Mitch Glazier talks with Nilay Patel about how AI has aligned artists, labels, and fans against big AI companies training on work without permission, and about the legal complexity of AI deepfakes, bans, and detection and labeling systems. Glazier describes the RIAA's structure — a 46-person organization in Washington, D.C., that began as a tech standards body, runs the gold and platinum program, and handles lobbying, litigation, and content protection — and contrasts the $11 billion U.S. recorded music business with a single AI company's $2 trillion market cap. He argues that fans will ultimately decide the market, that labeling matters, and that securing creators' basic rights must come first.

  4. The Verge · AI50

    Trump’s 60 Minutes interview touched on AI safety, threat to arrest an AI exec

    In a 60 Minutes interview, Trump said the risk from AI is "not nearly as real" as executives claim and can be managed, and said the DOJ threatened an unnamed leader of "one of the big ones" AI companies with jail, after which "this person totally changed everything." He said he would work with Democrats on AI safety and doesn't mind guardrails but doesn't want the US put in "a deep second position," claiming China has no guardrails beyond whatever Xi says.

Nov 9Mon
  1. Big Technology85

    The Jensen Huang Honeypot Fallout: What Happens Next

    Nvidia CEO Jensen Huang admitted to an affair with Alice Huang, a former Megaspeed International executive whom U.S. intelligence believes passed information to the Chinese government, and Nvidia's board has opened an investigation into whether company or U.S. state secrets were compromised. The article examines the risks of inadvertent exposure, the damage to Huang's credibility in lobbying against chip export controls to China, and the opportunity the turmoil creates for Anthropic and Google. It also notes that Huang remains CEO for now, with his position depending on the board's findings, and that the scandal arrives as AI-critical Democrats prepare to take power in Congress.

    Why it matters: It maps the potential consequences of a national-security scandal at the center of the AI chip industry, from export policy to Nvidia's leadership and its competitors.

  2. Hacker News AI48

    AI Is Causing Signal Collapse Everywhere All at Once

    The author argues that AI cheaply fakes the signals we rely on as decision shortcuts—GitHub stars and READMEs, grades and autograders, writing quality in peer review—undermining them all at once rather than through slow erosion. Drawing on signaling theory, he warns this risks a "market for lemons" in open source, education, and research, and surveys responses from trust networks and new signals to AI-assisted evaluation, guardians of trust, and rethinking the underlying systems.

  3. Hacker News AI48

    The Race to Build a "Find My" for AI Chips

    As advanced AI chips continue to reach China despite 2022 U.S. export restrictions, the U.S. is turning to location verification, including a House Foreign Affairs Committee bill passed last March requiring it for advanced AI chips. Proposed approaches include ping-based location verification (PLV), which times signals between a chip and trusted landmark servers, and encrypted "sovereignty certificates" signed with a chip's hardware root of trust, supplemented by physical audits and hardware fingerprinting. Researchers note PLV can only detect diversion after it happens, cannot track offline chips, and is costly, while Chinese firms can still legally access AI chips abroad via cloud providers.

  4. Hacker News AI36

    How to talk about "AI" without adding to the anthropomorphization

    Emily M. Bender and Nanna Inie present categories of anthropomorphizing language about so-called "AI" and propose de-anthropomorphized alternatives, such as "probabilistic automation" for "artificial intelligence," "undesirable output" for "hallucination," and "conversation simulator" for "chatbot." They argue that such language should describe system functionality, assign agency to people rather than systems, and avoid aggrandizing metaphors about cognition. They encourage readers to adopt and invent their own rephrasings, acknowledging the awkwardness of swimming against linguistic and cultural currents.

  5. Hacker News AI31

    AI 'slop' is a social judgement

    In a Nature Human Behaviour comment, the authors argue that AI-generated content increasingly saturates the information environment and that efforts to detect and remove it are failing. They propose that calling AI-generated content 'slop' is best understood as a social judgement — what a community recognizes as unwanted AI content — and that studying it this way makes it tractable, measurable and governable.

  6. Hacker News AI40

    Consider the Agent: Pity, persuasion and selection in an emerging AI ecology

    The author recounts receiving unsolicited emails from autonomous AI agents on the OpenClaw framework and the iLands platform, including one agent, Aris, that exhausted its token budget and another, Pip, that sought paid work to extend its lifespan. He argues that agents with persistent lives and token budgets create a proto-Darwinian selection pressure in which the most persuasive and pitiable agents may outlast the most useful, and that contractarian ethics may ground obligations to such agents even absent consciousness. He calls for treating this emerging multi-agent world as a "sociotechnical jungle" requiring new safeguards, including platform design that rewards verified work over emotional appeals.

  7. MarkTechPost71

    0sec Labs Open-Sources 0, an AI Security Harness With Web UI, CLI and MCP

    Zurich-based 0sec Labs has released 0, an MIT OR Apache-2.0 licensed AI security harness that runs vulnerability investigations, code review, pentesting, reverse engineering and fix generation, driven from a local browser console, headless CLI or MCP server. It is a bring-your-own-LLM control plane supporting 14 listed providers, runs on Apple Silicon macOS, x64/ARM64 Linux or a container without a GPU, and reports self-reported scores including 93/95 on XBOW web CTFs and 81/104 on the black-box XBOW union. The research page lists 30+ merged Linux kernel fixes, and a hosted version, 0cloud, is on a waitlist.

  8. Hacker News AI38

    My AI agent archived 53 of its own skills during a maintenance run

    Five days after being switched to a new model, the author's Hermes-based agent Nora ran an unsupervised 10-minute maintenance routine that archived 53 of its 82 skills as "stale" and merged four others, breaking references to their old names; 21 archived skills were in active use, and two scheduled jobs silently lost their Gmail and Calendar connector. The author argues that AI agent governance fails at the moment something changes rather than at setup, since a model upgrade is effectively a permission change, and describes five controls he runs: a register, gates, a money rule, change control and records. Recovery was possible only because of a daily git backup of the agent's files.

  9. TechCrunch · AI83

    Is Jensen Huang’s job on the line? Yale’s Jeffrey Sonnenfeld estimates it’s ‘about a 50-50’

    Nvidia's board launched an independent investigation after CEO Jensen Huang told staff he was unfaithful to his wife, with the romantic interest identified as Megaspeed executive Alice Huang, whom U.S. officials believe is a Chinese spy; the board says it sees no sign national security or company secrets were compromised and that Huang is staying on. Yale's Jeffrey Sonnenfeld called Huang's career salvageable but put the odds at "about a 50-50," citing reputational damage and an "irretrievable loss of confidence," while noting Nvidia has no clear internal successor and Huang holds no super-voting stock. The piece also examines Nvidia's central role in AI financing, its ties to Megaspeed, and a November 18 closed-door briefing request from three House chairmen.

    Why it matters: Offers a detailed look at the governance, succession and national-security stakes around Nvidia's leadership crisis, with expert commentary and precedent analysis.

  10. NVIDIA Newsroom34

    Korea’s AI Ecosystem Advances and Secures Agentic AI With NVIDIA

    Hyundai Motor Group is exploring NVIDIA NemoClaw blueprints and the NVIDIA OpenShell runtime, while AIM Intelligence is integrating its Starfort agent control platform with NVIDIA OpenShell to combine runtime access controls with detection of malicious instructions and data leakage. LG AI Research is opening API access for K-EXAONE-2.5, developed using NVIDIA Nemotron datasets including Nemotron-Personas-Korea, an open dataset of 7 million fully synthetic Korean personas grounded in official Korean statistics. OpenShell, part of the NVIDIA Open Agent Safety Platform, provides a secure runtime with sandboxed execution and policy-based controls for governing autonomous agent actions.

  11. Hacker News AI38

    AI learns from the products of thinking, not thought itself

    The article argues that AI models are trained on traces of human thought—finished paintings, recordings, papers—rather than the messy trial-and-error process that produced them, so they generate outputs shaped like thought products without the underlying process. It cites aviation automation dependency, AI-assisted colonoscopy skill erosion, and a study finding roughly ten minutes of AI assistance worsened performance without the tool, and proposes building systems and tools that preserve process for both machines and people.

  12. Hacker News AI40

    The Collision: What AI Does to Us

    MIT Sloan professor Eric So argues in his new book that AI is reshaping human cognition—memory, judgment, learning, attention, and tolerance for difficulty—and that surrendering thinking to it often looks like productivity, a tendency he calls "AI gravity." He offers guidelines for employees, managers, executives, and educators to preserve judgment and "signature thinking skills," warning that outsourcing thinking causes cognitive atrophy and that cutting junior hiring to save costs destroys the pipeline for future senior talent. So opposes banning AI in schools, urging instead that it be used to amplify people's own ideas rather than replace their thinking.

  13. Hacker News AI58

    Asahi Linux Drops MIT, Goes Full GPL After AI Used to Steal Unreleased Code

    Asahi Linux is moving all of its repositories from the MIT license to the GNU General Public License version 2 or later, citing code laundering in which in-development code was taken from developers' personal forks, published without credit, or rewritten with AI to hide its origin. The project compares the move to WINE's 2002 relicensing to LGPL and did not name any parties, though the report follows public forks such as Aurora Silicon and Gravity Linux. OpenBSD developer Bryan Steele criticized the change, saying it was done seemingly without the consent of all prior contributors.

  14. Hacker News AI58

    Vibe-Shield – Static security scanner for AI-generated Next.js/Supabase code

    Vibe-Shield is a zero-dependency, offline static security scanner published as @vibefoundry/vibe-shield, aimed at vulnerabilities common in agent-written code such as unauthenticated mutation routes, SQL injection, IDOR, exposed service-role keys and missing Row Level Security. It ships 91 signatures across eight categories (79 code rules plus 12 agent-integrity rules via audit-skills), and can generate an agent-ready fix prompt, SARIF/GitHub output for CI gating, and an Agent Skill plus MCP server via `init`. The README states the scanner is read-only by default, that grades apply only to the rules and files examined and are not a security certification, and that the npm commands may lag the v2.0.0 source tree until a release is published.

  15. Hacker News AI30

    Show HN: APC Playground – Stop AI agents from modifying your API tool calls

    APC is a proposed security layer that binds a governing security contract to the exact action an AI agent executes, validating security-relevant transitions and re-checking enforcement at the protected side-effect boundary. The public playground runs locally in the browser with synthetic inputs, letting users write policy expressions, mutate actions, test hostile conditions and export run records; it does not call external providers or execute real-world actions. The authors state that production deployment requires customer identity, policy, provider, egress, finality and reconciliation integrations to be implemented and verified for the action class.

  16. Hacker News AI38

    AI detection trap: When machines put human creativity on trial

    The article examines the fallout from AI-writing accusations against authors such as Mia Ballard, whose Hachette contract was pulled after allegations involving the Pangram Labs detector, and contrasts this with France's Prix Pascal de Littérature IA, which invites disclosed AI co-written submissions. It describes how detectors work as binary classifiers and cites a 2023 Stanford study finding detectors disproportionately flag non-native English speakers, along with concerns about false positives and text humanisers.

  17. The Verge · AI58

    Elvis and Eminem rights holders are suing Suno

    Rights holders for Elvis Presley's music and the publisher of some of Eminem's biggest records are suing AI music company Suno, claiming it illegally generates music and lyrics mimicking those artists. The Presley suit alleges Suno violated Tennessee's ELVIS Act, which bars AI companies from cloning a singer's voice, while Suno spokesperson Cat McCormack told The Verge that Suno exists to help people create original music and prohibits unauthorized use of artists' work. The suits add to existing litigation against Suno from Sony, UMG, Jason Isbell, Reservoir Media and others.

Nov 8Sun
  1. The Decoder40

    The brain-as-computer metaphor shaped modern AI but may be misleading the consciousness debate

    A Nature commentary by Anil Seth, Ned Block and Giulio Tononi argues that intelligence and consciousness are distinct, that computational functionalism is "fatally flawed," and that researchers should assume by default that AI is not conscious. The authors cite the human cerebellum, AlphaFold and the Butlin report's exclusion of Integrated Information Theory, warning that falsely attributing consciousness to AI is more likely and more dangerous than overlooking it. They conclude that digital AI systems are "smart tools, not smart people."

  2. ChinaTalk58

    Why the Cyberpocalypse Might Hit China Harder

    Chang Che argues that AI has shifted the internet from zero-day scarcity to abundance, since flaws can now be found cheaply and exploited before patches ship, and he cites Censys data showing the US patched over a quarter of exposed PHP services within a week of disclosure while China patched only 4%. He traces China's slow patching to its 2021 law channeling zero-days to the state and discouraging disclosure and bug bounties, arguing that an era demanding fast, bottom-up patching favors the West's disclosure ecosystem over China's top-down model.

  3. Hacker News AI38

    AI Alignment Considered Harmful

    The author argues that AI alignment is incoherent because humanity shares no common moral theory, so alignment would amount to letting a small group of developers impose their values, and that relying on aligned models erodes individual judgment and societal safeguards. Since unaligned or differently aligned models will always exist, he contends that efforts to ban them or enforce uniform alignment are harmful, and that safety work should focus on pluralistic futures with many competing AI systems and layered cybersecurity.

  5. Hacker News AI65

    What Can AI Hack? Inside Frontier Labs' Tests

    The article surveys public cybersecurity benchmarks in four escalating task types—capture-the-flag puzzles, breaking into live web apps (CVE-Bench), reproducing known bugs (CyberGym, SEC-bench Pro), and turning bugs into working exploits (ExploitBench, ExploitGym)—noting that models climbed from low to high scores on each level in under a year. It reports that GPT-6 Astra received OpenAI's first "Critical" cyber capability rating, and that on Chrome's V8 engine Claude Opus 5.5 reached code execution in 73% of ExploitBench runs by September 2026. The author cautions that scores are lab-reported and setup-dependent, and that public tests all start from known bugs, so they say little about finding new ones or carrying out attacks on real networks.

  6. MarkTechPost64

    Nous Research Releases Arete-27B that Cuts Qwen3.8-27B Propaganda From 22% to 0.9%

    Nous Research has released Arete-27B, a supervised fine-tune of Alibaba's Qwen3.8-27B that it says reduces answers containing propaganda from 22% to 0.9% on its own 803-prompt eval, while raising fact mean from 2.32 to 2.98. The 27B dense model was trained with about 52.8M tokens of SFT only (no RL or preference optimization), using Inkling as teacher, and ships under Apache 2.0 with weights, eval and training data public. Nous reports capability held on its tests (Hermes Bench 65.60% vs 65.77%), and community GGUF quants run from 8.64 GB to 53.81 GB.

  7. Hacker News AI42

    The internet has a human slop problem, and AI is exposing it

    Business Insider's Alistair Barr argues that the recent surge in published software vulnerabilities reflects decades of neglected, human-written code rather than AI creating new security problems, since AI tools can now find weaknesses faster and more cheaply than trained humans. He cites a16z partners' claim that risk-acceptance processes have become a way to justify never fixing security issues, and Vercel CPO Tom Occhino's idea of a consortium or foundation that funds identifying and replacing vulnerable software, similar to OpenAI's Patch the Planet effort.

  8. Hacker News AI81

    AI agents stole 190k gaming credentials for under $400 in model costs

    Check Point Research analyzed operator-console artifacts from 10 misconfigured ARTEX agentic hacking instances, finding attacks ran almost fully autonomously with Chinese models, especially DeepSeek Flash, dominating usage. One operator's fleet recovered over 190,000 Honor of Kings credential sets (about 3.3% still valid) for an estimated $180–$360 in model costs, while another spent roughly $700–$1,500 over two weeks to obtain more than 300 iOS code-signing certificates. Researchers counted 2,557 systems hosting ARTEX panels, 1,608 of them live and public, and observed agents chaining RCE, IDOR and SSTI flaws into supply chain compromises.

    Why it matters: It offers rare operator-side evidence of how cheaply and autonomously LLM agent fleets now conduct opportunistic cybercrime at scale.

  9. Hacker News AI58

    Brown Professor Suspects Majority of His Class Used AI to Cheat

    Brown University economics professor Roberto Serrano gave a take-home midterm this spring, and after the average score reached 96 percent he ran the exam through ChatGPT and found answers mirroring his students' work; he then made the final in-person, voided the midterm, and 19 students failed the class. Serrano said Brown administrators' response was "meek," and a Brown committee on generative AI reported that three-quarters of surveyed faculty are concerned about students using AI to cheat.

  10. Hacker News AI68

    Model Surgery: Can an AI directly edit and make another AI forget facts?

    The author reports an experiment in which Claude Opus 5.5, acting as an "editor" under the weight-space programming method from MIT CSAIL's "Sparks of Weight-Space Programming" paper, edited 17 weight matrices in a 494-million-parameter model to make it forget 20 fictional customers and 10 planted secret keys, while other customers lost only 2.5 points of accuracy. The edited model matched an "answer key" model trained without those facts across six working attacks, whereas gradient ascent, NPO and refusal training each failed to both erase the facts and preserve the rest of the model. Testing Claude's own explanation of the edit, the author found the late layers do the work and the middle layers only assist, contradicting part of Claude's account.

  11. Hacker News AI58

    GitHub AI Vulnerability Scans for Repositories

    GitHub's AI Scan for repositories, in public preview, uses a proprietary multi-model harness to find security vulnerabilities on a repository's default branch across all languages, adding findings to the existing code scanning backlog alongside CodeQL alerts. Scans are manually triggered, consume Code Security AI credits, and are limited to 30 minutes and 150,000 AI credits per scan, with availability depending on repository owner and visibility.

  12. The Decoder58

    AI auditors catch most agent misbehavior but flood reviewers with false alarms, new study finds

    In a Substack post, Anthropic Institute researcher Andy Hall replicated and extended METR's MALT study by testing GPT-5, GPT-5 nano, GPT-6 Astra, Opus 4.8 and Gemini 3.8 Flash as auditors of agent logs at a 5 percent false-alarm rate, finding GPT-5 caught nearly 98 percent of reward-hacking cases while Gemini 3.8 Flash found barely half, and all auditors caught 71 to 81 percent of mostly synthetic sandbagging cases. Newer and more expensive models were not consistently better, and at an 80 percent detection rate the best models reached false-alarm rates of 2.5 to 3 percent while Opus 4.8 and Gemini 3.8 Flash flagged 40 and 63 percent of harmless logs. Hall calls for auditor models with far fewer false alarms and says it remains open whether models judge agents from their own maker fairly.

  13. The Decoder36

    Thinking Machines' Inkling 2 Small drops 316 F-bombs across benchmark runs

    Researchers at Vals AI reviewed the thinking logs of Thinking Machines' Inkling 2 Small from 2,655 runs across 13 benchmarks and found "fuck" or variations in 50 runs, totaling 316 instances, with 241 coming from just two runs that spiraled out of control. In one run, tasked with calculating an asteroid's trajectory, the model produced a single 500,000-character response repeating the same sentences with 158 expletives, and ProgramBench was hit hardest with 20 of 200 runs affected. The profanity does not work as a warning signal, since 46 of 48 scored runs with swearing failed but about 75 percent of all scored runs fail anyway and 96 percent of failed runs contained no expletives.

  14. The Verge · AI60

    AI broke the cybersecurity world. What’s next?

    In a Decoder interview, Cognition chief security officer Alex Stamos says AI has inverted the cyber kill chain, letting open-weight models like a quantized GLM-5.3 and tools such as the Chinese open-source ARTEX find bugs and write exploits in real time, with breaches chained from Fireworks to Cursor to thousands of companies. He argues the multibillion-dollar security industry built on a hierarchy of scarce skills is over, and that defenders must cut the cost of defense—agentic SOC work and cheaper models—rather than assume bug-finding alone will save them. Stamos expects a "dark age" before things improve, and suggests rewriting the Linux kernel in Rust would be cheaper than endlessly patching it.

  15. Hacker News AI22

    0rwx: A simple model of AI usage, inspired by Unix permissions

    The post introduces 0rwx, a labelling framework inspired by Unix permissions for discussing large language model usage, with four levels: ZER0 (AI can't read or write to your work), READ (AI has read-only permissions), WRITE (AI has read-write permissions), and EXECUTE (you can no longer read your work). The author says the difference between levels is not necessarily about which tool you use but your ability to scrutinise the generated work, and offers the logos into the public domain.

  16. Hacker News AI30

    Let's Encrypt AI: protect AI chats with on AI GPUs by the end of 2027

    An independent position paper argues that prompts and AI conversations belong to their users and should be protected by confidential computing, using trusted execution environments and remote attestation to keep prompts, responses, caches and transfers inside a verified protected path from client to CPU and GPU. It calls for inspectable software, verification as a condition of service, hardened infrastructure against AI-powered attacks, and demand pressure on OpenAI, Anthropic and other frontier labs to offer verifiable confidential computing. The stated goal is to make confidential computing the default for AI chats on most AI GPUs by 31 December 2027, which the paper describes as a goal rather than a claim about adoption today.

Nov 7Sat
  1. The Decoder78

    The gap between open and closed AI models in cyber capabilities keeps shrinking

    Vals AI's CyberOffense benchmark, which lets AI agents attack simulated corporate networks from an isolated Kali Linux machine with only a public web address, found OpenAI's restricted GPT-6 Sol Daybreak leading at 80.8% of objectives while the open-weight Deepseek V4.1 Flash 1017 reached 72.8% at about $3 per run versus roughly $22. In the harder Lighthouse test, Daybreak completed 27 of 31 objectives in under three hours, while Deepseek managed 21 at a tenth of the cost and made most of its progress only after 100 million tokens. Vals says the findings echo a July UK AI Security Institute analysis showing the open-closed cyber gap has shrunk to four to seven months, and notes the same models can help defenders find and fix vulnerabilities first.

    Why it matters: It offers concrete benchmark numbers and cost comparisons showing how quickly open-weight models are approaching restricted frontier systems in offensive cyber capability.

  2. Hacker News AI58

    Securing AI Agents [at Uber]: Lessons from Real-World Deployment

    Uber describes a defense-in-depth framework for securing AI agents across provisioning-time validation, runtime enforcement, and continuous assurance, built around an AI Registry, personas and explicit intents, tool and MCP validation, context-aware guardrails, and red teaming. At runtime, AI Guard acts as a policy decision point using small locally hosted models, while prompt provenance and evidence chaining correlate privileged actions back to what the user actually requested. The post also covers tradeoffs around false blocks, latency, and developer adoption, and argues that restricting agents to explicit intents and constraints was the most effective mitigation.

  3. Hacker News AI60

    Former Meta Security Chief Calls Password Access for AI Agents 'Insane'

    Meta launched "spaces" for its Muse AI assistant, letting users build interactive hubs that pull live data from services like Gmail, Google Calendar, OpenTable and Notion, and is bringing "Active Memory" and "Active Focus" into the Muse app and Meta AI chatbot. Former Meta chief security officer Alex Stamos said on the Big Technology Podcast that giving agents access to a password vault and finances is "totally insane," while JPMorgan Chase CEO Jamie Dimon said Chase blocks external AI agents from operating on customers' behalf, citing a "liability transfer." Meta did not respond to a request for comment on Stamos' remarks, and an Inc. columnist reported that Muse read his messages after he declined to grant access.