Blackbox Blast
BB Preflight
← Incident log

Weekly debates

World AI Debates

How the world is preparing for AI incidents and runaway systems. Policy, regulation and safety-research arguments, presented as their authors make them. Blackbox Blast takes no side.

2026-W45

1 debate

Safety researchJapanese

How much more often were AI-generated personalized phishing emails clicked than generic ones?

ITmedia NEWS · by Yamashita (Seamless)

Claim

[Needs review] This source is a news article summarizing the experimental results of a research paper, not someone's argument about policy or regulation. According to the article, researchers at the Technical University of Berlin and other institutions report that spear-phishing emails generated per person by an LLM had a 10% click rate, 2.74 times the 3.9% of generic phishing that sends everyone the same text.

Reasoning

Per the article, the team ran an experiment on 7,741 university staff members. They put each person's email address into a search engine, extracted career and affiliation details from public web pages, and used several linked AI models to build a personal profile and automatically generate a sender and message suited to that person.

Information was found online for 4,010 people, and personalized emails were sent to 3,310 of them. Spear phishing written by hand by experts had a higher click rate of 24.2%, but the article notes that manual work takes enormous time and effort, whereas the LLM took about 50 hours to write about 4,000 emails, at about $0.03 per email.

People with more public information were harder to fool, while the low and medium groups were easier to fool. The university's spam filter flagged only 1 of 3,949 emails as suspicious. The article presents the paper as published at the 2026 USENIX Security Symposium.

Why it matters
In the article's results, the university's filter stopped almost none of the personalized emails. An operation that leaves email defense to filters alone may need a second look.

2026-W44

1 debate

OtherJapanese

Should operators prepare for two critical PHP vulnerabilities before the fix is released?

note · by AK.Ita

Claim

[Needs review] This article is an explainer on PHP vulnerabilities, not a debate about AI agents or AI policy. Check whether it can be treated as a debate item. The author, AK.Ita, points to the advisory that IPA issued before the PHP fix was released, and argues that operators should not wait for the fix but get ready in advance so they can respond as soon as it is published.

Reasoning

According to the author, the PHP security team announced on October 20 that it would release a critical fix at 23:00 JST on October 21. The targets are CVE-2026-109618 and CVE-2026-109619, both rated Critical at 9.2 on CVSS v4.0. The author notes that CERT-EU warns they could lead to unauthenticated remote code execution.

At the same time, the concrete attack method and conditions have not been made public, and the author says the danger should be kept separate from confirmed facts. External reports mention signs of attack, but IPA states explicitly that it has not itself confirmed attacks exploiting these two flaws, as the author sums it up.

The author also says WordPress and PHP are separate things, so even with WordPress core and plugins up to date, the server-side PHP may need separate handling. As preparation the author lists identifying systems that use PHP, checking the PHP version, confirming who applies updates and how, and reviewing access logs and similar records, and notes that CERT-EU suggests considering isolating internet-facing applications if a fix cannot be applied right away.

Once the fix is released, attackers can also study the changes, and not every server updates at the same time, so preparing for that gap is worthwhile, the author argues.

Why it matters
If you only start investigating after the fix is out, you cannot respond right away. Before release, work out where PHP is used in your environment and who will apply the update, and by what steps.

2026-W43

1 debate

OtherJapanese

Can an open-source autonomous AI intrusion tool be recalled once its release is stopped?

X (post by t-Kimura) · by t-Kimura (@t__kimura)

Claim

t-Kimura sees ARTEX, an open-source autonomous AI penetration-testing system, as a real example of an attack-side orchestrator that ties together multiple LLMs and agents. He argues that the capability has already spread even though its author stopped publishing it, so withdrawing it is too late as an after-the-fact response.

Reasoning

According to the author, ARTEX was released by a developer called Autumn and consists of a Planner that decides the direction of the attack and several Workers that do the actual work. The LLMs split the tasks of analyzing targets, planning intrusion paths, executing commands and inspecting HTTP traffic, and decide their next step from the assets and vulnerabilities they have found.

The developer also built the reconnaissance tool ScopeSentry, and ARTEX can sync asset information from it, so the whole chain from reconnaissance to intrusion is available free from one developer's tools, he says.

He writes that the main backend is DeepSeek, with GLM-5.3 and Grok 4.6 as auxiliaries, and that the attacker also operated Claude Code directly. He says DeepSeek access is presumed to go through an API reseller, and he reads this as a typical filter-evasion pattern: hiding the origin of access and mixing in models with weaker safeguards.

He also covers actual use against South Korean financial institutions. According to him, the case came to light because an open directory on a Hong Kong IP exposed Claude Code session history, a memory file and ARTEX configuration files.

At least nine banks were targeted between late September and early October. Shinhan (about 25,000 records) and KB Kookmin (119) each announced their own figures, and he notes that the total varies by report and is not confirmed.

Around October 8, the developer made ARTEX closed-source and issued a statement saying it is unrelated to the Korean case and that updates and maintenance will stop. The author says this does not directly rebut CrowdStrike's analysis and only denies a connection.

He points out that mirrors, restored copies and a Korean-language version are already circulating, and that forks are legal under AGPL-3.0. He then names three core threats: the barrier to entry has nearly vanished; the Planner + Workers design runs on its own without step-by-step human instruction, letting one person attack many targets at once; and a capability released as open source can later be stopped neither by its author's will nor by national regulation.

These are his grounds. Note that this is one post giving the author's own assessment, and the attribution and damage figures in it are described there as estimated or unconfirmed.

Why it matters
Stopping distribution of a tool does not end the threat while copies and derivatives remain. Plan your vulnerability management on the assumption that tools able to run multi-stage attacks autonomously will be used against you.

2026-W42

1 debate

OtherJapanese

Are AI agents already being used to attack Japanese websites?

Scutum WAF Tech Blog · by Kanatoko (Scutum developer; director, BitForest Co., Ltd.)

Claim

Kanatoko, developer of the cloud WAF Scutum and a director of BitForest Co., Ltd., argues that the stage of debating whether AI agents are being used to attack Japanese websites is over, and predicts that nearly all attackers will come to use AI agents. He adds that defenses do not change much depending on whether an attacker uses an AI agent.

Reasoning

According to the author, Scutum began monitoring on the night of October 10 with a mechanism added to its WAF to tell whether an attack source is an AI agent, and by the 14th it had confirmed attacks on Japanese companies' websites from at least three IP addresses.

He does not disclose the details, to keep attackers from countering it, but says it turns AI agents' traits against them, such as reading the server's responses and adjusting their next action; in a development environment it detected all eight recent high-performance models tried, and when it does detect, it is almost never wrong.

Of the three, one ran only about 1,000 scans, one switched among several tools across five stages ending in repeated contact-form submissions, and the last shifted through ten stages over nearly eight hours, including high-speed scans, attacks on known vulnerabilities and login brute-forcing.

As defenses, he names eliminating vulnerabilities in advance, including in backend APIs, and, where needed, deploying a WAF like Scutum that does not rely on signatures alone.

Why it matters
The author's view is that defenders' work does not change much when attackers use AI agents. Still, an opponent that reads responses and keeps changing tactics for hours raises the stakes of leaving no vulnerability, including in contact forms, login pages and the APIs behind them.

2026-W41

1 debate

Regulation

Can the EU AI Act stop a dangerous AI model quickly enough?

AI Policy Bulletin · by Eliška Andrš (Future of Life Institute)

Claim

Eliška Andrš, a policy researcher at the Future of Life Institute, credits the EU AI Act with setting clear safety standards and clear conditions for intervention by the European Commission, and argues that the Commission should prepare now so it can act swiftly against a dangerous model.

Reasoning

She points to June, when two new Anthropic models were briefly suspended to comply with an emergency export control order that the US government reportedly issued.

The Act requires providers of general-purpose AI models to evaluate them, assess and mitigate risks and report serious incidents, and lets the Commission remove a model from the market as a last resort. A July 2026 implementing regulation also allows interim measures in urgent cases, and in such cases the Commission can act without first hearing the provider.

Using those powers, she argues, takes the will to act despite geopolitical and economic pressure, and a well-resourced pool of evaluators who can spot the risks quickly.

Why it matters
A model you depend on can become unavailable in a region overnight because of a regulator's decision. If your operation relies on a single model, decide on the fallback before it stops.