#prog #ml #article
Thoughts on slowing the fuck down
Thoughts on slowing the fuck down
There's a much more important difference between clanker and human. A human is a bottleneck. A human cannot shit out 20,000 lines of code in a few hours. Even if the human creates such booboos at high frequency, there's only so many booboos the human can introduce in a codebase per day. The booboos will compound at a very slow rate. Usually, if the booboo pain gets too big, the human, who hates pain, will spend some time fixing up the booboos. Or the human gets fired and someone else fixes up the booboos. So the pain goes away.
With an orchestrated army of agents, there is no bottleneck, no human pain. These tiny little harmless booboos suddenly compound at a rate that's unsustainable. You have removed yourself from the loop, so you don't even know that all the innocent booboos have formed a monster of a codebase. You only feel the pain when it's too late.
👍13
#itsec #ml #suckassstory #article
The Boy That Cried Mythos: Verification is Collapsing Trust in Anthropic
Вы, вероятно, слышали в недавних новостях о модели Mythos от Anthropic и о том, насколько она хороша в поиске уязвимостей... Со слов самих Anthropic.
Эксперт по компьютерной безопасности Davi Ottenheimer решил проверить, насколько заявления о возможностях Mythos соответствуют правде.
TL;DR: даже собственный документ Anthropic даёт понять, что их инструмент, мягко говоря, не соответствует заявленным возможностям. Ниже мои отдельные цитаты, но я крайне рекомендую прочитать статью целиком.
(^проверено в AISLE)
Автор также утверждает, что Anthropic на волне хайпа отвоёвывает себе тёплое местечко под солнцем в обход существующих ограничений.
The Boy That Cried Mythos: Verification is Collapsing Trust in Anthropic
Вы, вероятно, слышали в недавних новостях о модели Mythos от Anthropic и о том, насколько она хороша в поиске уязвимостей... Со слов самих Anthropic.
Эксперт по компьютерной безопасности Davi Ottenheimer решил проверить, насколько заявления о возможностях Mythos соответствуют правде.
TL;DR: даже собственный документ Anthropic даёт понять, что их инструмент, мягко говоря, не соответствует заявленным возможностям. Ниже мои отдельные цитаты, но я крайне рекомендую прочитать статью целиком.
I’ve been getting more and more curious about the risk from Anthropic’s Claude Mythos Preview. So I pulled the system card, a whoppingly inefficient 244-page document that devotes just seven pages to the claim that the model is too dangerous to release
The flagship demonstration of “unprecedented cyber capability” is in fact a model that weaponized two bugs that a different Anthropic model had already found, in software Mozilla had already patched, in a harness with the actual defenses turned off, where the “triage” step it performed is also performed by its predecessor.
The bugs Anthropic used to justify a $100 million consortium, eleven Fortune-100 partners, a “too dangerous to release” decision, and global headlines that “frightened the British” — an open-weights 3.6B-parameter model finds them too, for eleven cents per million tokens.
(^проверено в AISLE)
The Mythos system card tested the model against small-scale enterprise networks with no active defenses and the model succeeded. The same document tested the model against a properly configured sandbox with modern patches and the model failed.
Автор также утверждает, что Anthropic на волне хайпа отвоёвывает себе тёплое местечко под солнцем в обход существующих ограничений.
By withholding Mythos from general release and granting access only through the Glasswing consortium — Apple, Google, Microsoft, Amazon, Broadcom, Cisco, CrowdStrike, JPMorganChase, Nvidia, Palo Alto Networks, the Linux Foundation — Anthropic inserts itself as a de facto clearance-granting body for an “uplift” of vulnerability knowledge. Without a statutory basis. Without congressional oversight. Without FOIA exposure. Without a neutral arbiter. With a partner list drawn entirely from the largest incumbents in the industry it claims to be protecting.
<...>
That is not a safety posture. It’s regulatory capture dressed as restraint. And it is being constructed with no democratic input, in a legal vacuum, by a private company whose business model depends on selling access to the very capability it has declared too dangerous to release.
👍5🤣3❤1🥰1😢1💩1
#ml #article
AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights (PDF)
Из интересного: предпочтение собственной генерации коррелирует со способностями LLM к распознанию собственного вывода, которая, в свою очередь, коррелирует с количеством весов. Описанные авторами "simple interventions" включают в себя два способа. Первый — явное включение в промпт указание на игнорирование авторства текста. Второй — использование ансамбля LLM, включающих в себя модели с более слабыми способностями к самораспознаванию.
AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights (PDF)
Using a large-scale controlled resume correspondence experiment, we find that LLMs consistently prefer resumes generated by themselves over those written by humans or produced by alternative models, even when content quality is controlled. The bias against human-written resumes is particularly substantial, with self-preference bias ranging from 67% to 82% across major commercial and open-source models. To assess labor market impact, we simulate realistic hiring pipelines across 24 occupations. These simulations show that candidates using the same LLM as the evaluator are 23% to 60% more likely to be shortlisted than equally qualified applicants submitting human-written resumes, with the largest disadvantages observed in business-related fields such as sales and accounting. We further demonstrate that this bias can be reduced by more than 50% through simple interventions targeting LLMs' self-recognition capabilities.
Из интересного: предпочтение собственной генерации коррелирует со способностями LLM к распознанию собственного вывода, которая, в свою очередь, коррелирует с количеством весов. Описанные авторами "simple interventions" включают в себя два способа. Первый — явное включение в промпт указание на игнорирование авторства текста. Второй — использование ансамбля LLM, включающих в себя модели с более слабыми способностями к самораспознаванию.
arXiv.org
AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
As artificial intelligence (AI) tools become widely adopted, large language models (LLMs) are increasingly involved on both sides of decision-making processes, ranging from hiring to content...
🤔4🌚2
#prog #ml #abnormalprogramming
Llama.ttf
Есть ссылка на видео с демонстрацией. Сам шрифт весит 60 мегабайт, по очевидным причинам.
(thanks @programming_sucks)
Llama.ttf
The font shaping engine HarfBuzz, used in applications such as Firefox and Chrome, comes with a Wasm shaper allowing arbitrary code to be used to "shape" text.
In particular, this "arbitrary" code could in principle be an entire LLM inference engine with trained parameters bundled inside, relying on treating text containing magic symbols for fake "ligatures" to initialize the LLM and use it to generate text.
It could also in principle be an entire LLM inference engine (Llama in our case, hence the name) except instead of only being in principle it's what this is.
Есть ссылка на видео с демонстрацией. Сам шрифт весит 60 мегабайт, по очевидным причинам.
(thanks @programming_sucks)
fuglede.github.io
llama.ttf
llama.ttf is a font file which is also a large language model and an inference engine for that model.
🤯11❤3😁2