Next Newsletter

Important concepts in numerical linear algebra and related areas - higham/what-is: Important concepts in numerical linear algebra and related areas

Like, if a word is a party, and letters are guests, which letters want to hang out with each other at the party?

72% of U.S. adults say tipping is expected in more places today than it was five years ago. But even as Americans say they’re being asked to tip more often, only about a third say it’s extremely or very easy to know whether (34%) or how much (33%) to tip for various services.

A stunning advancement in a long history of stargazing, the James Webb telescope reveals light where once we saw only darkness. Our view of the universe will never be the same.

A series of prompts to learn bash commands based on the circles of hell from Dante's Inferno - f23-bzan545/dantes-inferno: A series of prompts to learn bash commands based on the circles of...

Hunt down the AirbnBandit armed with a SQL database, a series of clues, and you're own ingenuity!

How fast can you name 10 SQL keywords?

Distilling the world's data, information & knowledge into beautiful infographics & visualizations

Internet speeds have come a long way since the days of the dial-up modem, but sometimes you can’t beat the millennia-old method of carrier pigeon.

Model licensing & surveillance will likely be counterproductive by concentrating power in unsustainable ways

A practical guide to using embeddings to solve common product engineering problems, or, what you can do when computers can understand what things mean.

Weekly bite-sized tips on DataViz, Shiny and Stats/Machine Learning.

43 votes and 9 comments so far on Reddit

A year ago, Meta's Galactica was released — just two weeks before OpenAI's ChatGPT. It survived for only three days, but its legacy endures.

The story of how one independent researcher conducted the largest-ever survey on fetishes, and what it has to teach us about sex, pleasure, and social science methodology.

AI image generators like Stable Diffusion and DALL-E amplify bias in gender, race and beyond, despite efforts to detoxify the data fueling these results.

The Times used video evidence, data and interviews to reconstruct the wildfire that raged through what was once Hawaii’s royal capital.

These are income ranges for the ten most common jobs at different income levels.

All the junk orbiting the Earth could disrupt life down here. The world has some options for dealing with it.

Weekly bite-sized tips on DataViz, Shiny and Stats/Machine Learning.

Write, test, document, and distribute software packages and technical articles — all in one place, your notebook.

h20: TLDR/h2 pI examined all the biorisk-relevant citations from a policy paper arguing that we should ban powerful open source LLMs./p pNone of them provide good evidence for the paper's conclusion. The best of the set is evidence from statements from Anthropic -- which rest upon data that no one outside of Anthropic can even see, emand/em on Anthropic's interpretation of that data. The rest of the evidence cited in this paper ultimately rests on a single extremely questionable "experiment" without a control group./p pIn all, citations in the paper provide an illusion of evidence ("look at all these citations") rather than actual evidence ("these experiments are how we know open source LLMs are dangerous and could contribute to biorisk")./p pA recent emfurther/em paper on this topic (published emafter/em I had started writing this review) continues this pattern of being more advocacy than science./p pAlmost all the bad papers that I look at are funded by Open Philanthropy. If Open Philanthropy cares about truth, then they should stop burning the epistemic commons by funding "research" that is always going to give the same result no matter the state of the world./p h21: Principles/h2 pWhat emcould/em constitute evidence that powerful open-source language models contribute or will contribute substantially to the creation of biological weapons, and thus that we should ban them?/p pThat is, what kind of anticipations would we need to have about the world to make that a emreasonable thing to think/em? What other beliefs are a necessary part of this belief making any sense at all?/p pWell, here are two pretty-obvious principles to start out with:/p ul li pemPrinciple of Substitution/em: We should have evidence of some kind that the LLMs can (or will) provide information that humans cannot also easily access through other means -- i.e., through the internet, textbooks, YouTube videos, and so on./p /li li pemBlocker Principle/em: We should have evidence that the emlack of information/em that LLMs can (or will) provide is in fact a significant blocker to the creation of bioweapons./p /li /ul pThe first of these is pretty obvious. As example: There's no point in preventing a LLM from telling me how to make gunpowder, because I can find out how to do that from an encyclopedia, a textbook, or a novel like Blood Meridian. If you can substitute some other source of information for an LLM with only a little inconvenience, then an LLM does not contribute to the danger./p pThe second is mildly less obvious./p pIn short, it could be that emmost of/em the blocker to creating an effective bioweapon is not knowledge -- or the kind of knowledge that an LLM could provide -- but something else. This "something else" could be access to DNA synthesis; it could be the process of culturing a large quantity of the material; it could be the necessity of a certain kind of test; or it could be something else entirely./p pYou could compare to atomic bombs -- the chief obstacle to building atomic bombs is a href="https://www.theguardian.com/world/2003/jun/24/usa.science"probably not/a the actual knowledge of how to do this, but access to refined uranium. Thus, rather than censor every textbook on atomic physics, we can simply control access to refined uranium./p pRegardless, if this other blocker constitutes 99.9% of the difficulty in making an effective bioweapon, and lack of knowledge only constitutes 0.1% of the difficulty, then an LLM can only remove that 0.1% of the difficulty, and so open source LLMs would only contribute marginally to the danger. Thus, bioweapons risk would not be a good reason to criminalize open-source LLMs./p p(I am not speaking theoretically here -- a a href="http://philsci-archive.pitt.edu/22539/"paper/a from a researcher at the Future of Humanity Institute argues that the actual product development cycle involved in creating a bioweapon is far, far more of an obstacle to its creation than the basic knowledge of how to create it. This is great news if true -- we wouldn't need to worry about outlawing open-source LLMs for this reason, and we could perhaps use them freely! Yet, I can find zero mention of this paper on LessWrong, EAForum, and even on the Future of Humanity Insitute website. It's surely puzzling for people to be so indifferent about a paper that emmight/em free them from something that they're so worried about!)/p pThe above two principles -- or at the strongvery least the first/strong -- are the emminimum/em for the kind of things you'd need to consider to show that we should criminalize LLMs because of biorisk./p pArguments for banning open source LLMs that do not consider the emalternative/em ways of gaining dangerous information are entirely non-serious. And arguments for banning them that do not consider emwhat role LLMs play in the total risk chain/em are only marginally more thoughtful./p pAny emactually good/em discussion of the matter will not end with these two principles, of course./p pYou need to also compare the good that open source AI would do against the likelihood and scale of the increased biorisk. The a href="https://en.wikipedia.org/wiki/2001_anthrax_attacks"2001 anthrax attacks/a killed 5 people; if open source AI accelerated the cure for several forms of cancer, then even a hundred such attacks could easily be worth it. Serious deliberation about the actual costs of criminalizing open source AI -- deliberations that do not rhetorically minimize such costs, shrink from looking at them, or emphasize "other means" of establishing the same goal that in fact would only do 1% of the good -- would be necessary for a policy paper to be a empolicy paper/em and not a puff piece./p p(Unless I'm laboring beneath a grave misunderstanding of what a policy paper is actually intended to be, which is a hypothesis that has occurred to me more than a few times while I was writing this essay.)/p pOur current social deliberative practice is bad at this kind of math, of course, and immensely risk averse./p h22: "Open-Sourcing Highly Capable Foundation Models"/h2 pAs a proxy for the general "state of the evidence" for whether open-source LLMs would contribute to bioterrorism, I looked through the paper a href="https://cdn.governance.ai/Open-Sourcing_Highly_Capable_Foundation_Models_2023_GovAI.pdf""Open-Sourcing Highly Capable Foundation Models"/a from the a href="https://www.governance.ai/"Center for the Governance of AI/a. I followed all of the biorisk-relevant citations I could find. (I'll refer to this henceforth as the "Open-Sourcing" paper, even though it's mostly about the opposite.)/p pI think it is reasonable to treat this as a proxy for the state of the evidence, because lots a href="https://twitter.com/mealreplacer/status/1707905578150908252"of/a a href="https://twitter.com/jonasschuett/status/1707846395246354691"AI/a a href="https://twitter.com/NoemiDreksler/status/1707780982974140875"policy/a a href="https://twitter.com/S_OhEigeartaigh/status/1707763424799678674"people/a specifically praised it as a good and thoughtful paper on policy./p pThe paper is nicely formatted PDF, with abstract-art frontispiece and nice typography; it looks serious and impartial; it clearly intends policy-makers and legislators to listen to its recommendations on the grounds that it emprovides actual evidence/em for its recommendations./p pThe paper specifically mentions the dangers of biological weapons in its conclusion that some highly capable foundation models are just too dangerous to open source. It clearly emwants/em you to come away from reading the paper thinking that "bioweapons risk" is a strong supporting pillar for the overall claim "do not open source highly capable foundation models."/p pThe paper is aware of the substitutionary principle: that LLMs must be able to provide information not available though other means, if bioweapons-risk is to provide any evidence that open-sourcing is too dangerous./p pThus, it several times alludes to how foundation models could "reduce the human expertise" required for making bioweapons (p13) or help relative to solely internet access (p7)./p pHowever, the paper does strongnot/strong specifically make the case that this is true, or really discuss it in any depth. It instead emsimply cites other papers/em as evidence that this is -- or will be -- true. This is in itself very reasonable -- if those other papers in fact provide emgood experimental evidence/em or even emtight argumentative reasoning/em that this is so./p pSo, let's turn to the other papers./p pThere are three clusters of citations, in general, around this. The three clusters are something like:/p ul liBackground information on LLM capabilities, or non-LLM-relevant biorisks/li liAnthropic or OpenAI documents / employees/li liPapers by people who are ostensibly scientists/li /ul pSo let's go through them in turn./p pstrongImportant note/strong: I think one way that propaganda works, in general, is through a href="https://en.wikipedia.org/wiki/Brandolini%27s_law"Brandolini's Law/a -- it takes more energy to explain why bullshit is bullshit than to produce bullshit. The following many thousand words are basically an attempt to explain why the all of the citations about emone/em topic in a single policy paper are, in fact, propaganda, and are a facade of evidence rather than evidence. My effort is thus weakened by Brandolini's law -- I simply could not examine emall/em the citations in the paper, rather than only the biorisk-relevant citations, without being paid, although I think they are of similar quality -- and I apologize for the length of what follows./p h23: Group 1: Background material/h2

Seven years, 60 countries, 935 internet shutdowns: How authoritarian regimes found an off switch for dissent.

Google is analyzing data from its Maps app to suggest how cities can adjust traffic light timing to cut wait times and emissions. The company says it’s already cutting stops for millions of drivers.

Scoring a table in New York can seem impossible, but may be easier than you think. Here’s how the competition played out for one dinner, with tips from the winning players.

The Bird Migration Explorer is your guide to the heroic annual journeys made by over 450 bird species, and the challenges they face along the way.

We looked at over 1,400 romance novel covers featured in Publishers Weekly from 2011 to 2023 and evaluated each cover based on its raunchiness (or level of undress), art style, and representation of racial diversity.

If you wear women's clothes, you already know this. But now we've got the data to show it.

Taiwan is about to enter the super-aged era