Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
OpenEQA: From word models to world models
OpenEQA combines challenging open-vocabulary questions with the ability to answer in natural language. This results in a straightforward benchmark that demonstrates a strong understanding of the environment—and poses a considerable challenge to current foundational models. We hope this work motivates additional research into helping AI understand and communicate about the world it sees.
(PDF) Ethics of Quantum Technologies: A Scoping Review
The majority of the research has focused on the potential impact of quantum technologies on privacy and security, the potential impact of quantum technologies on the trust of those systems, and the potential for creating new forms of inequality in access to the technology.
Anil, C., Durmus, E., Sharma, M., Benton, J., Kundu, S., Batson, J., ... & Duvenaud, D. (2024). Many-shot Jailbreaking.
Long contexts represent a new front in the struggle to control LLMs. We explored a family of attacks that are newly feasible due to longer context lengths, as well as candidate mitigations. We found that the effectiveness of attacks, and of in-context learning more generally, could be characterized by simple power laws. This provides a richer source of feedback for mitigating long-context attacks than the standard approach of measuring frequency of success
Does Transformer Interpretability Transfer to RNNs?
Codegemma report
ReALM: Reference Resolution As Language Modeling
Jamba: A Hybrid Transformer-Mamba Language Model
Nay, J. J., Karamardian, D., Lawsky, S. B., Tao, W., Bhat, M., Jain, R., ... & Kasai, J. (2024). Large language models as tax attorneys: a case study in legal capabilities emergence. Philosophical Transactions of the Royal Society A, 382(2270), 20230159.
🥇Top ML Papers of the Week
Long-form factuality in large language models
Cultural Bias in Explainable AI Research: A Systematic Analysis | Journal of Artificial Intelligence Research
Language Models Can Reduce Asymmetry in Information Markets
Avery, J. J., Abril, P. S., & del Riego, A. ChatGPT, Esq.: Recasting Unauthorized Practice of Law in the Era of Generative AI. Yale Journal of Law & Technology, 26(1).
Standardized nomenclature for litigational legal prompting in generative language models
Scientific Educations Among U.S. Judges
Modeling interconnected social and technical risks in open source software ecosystems - William Schueller, Johannes Wachs, 2024
Grady Booch on Twitter / X
Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
Download PDF
Chewing Gum Is Associated with Better Diet Quality but Not Oral Health Measures in U.S. Adults
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
Investigating Continual Pretraining in Large Language Models: Insights and Implications
Download PDF
AutoEval Done Right: Using Synthetic Data for Model Evaluation
Download PDF
Data Interpreter: An LLM Agent For Data Science
Simultaneous and Heterogenous Multithreading | Proceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture
PDF
🥇Top ML Papers of the Week
Rosen, D., Oh, Y., Chesebrough, C., Zhang, F. Z., & Kounios, J. (2024). Creative flow as optimized processing: Evidence from brain oscillations during jazz improvisations by expert and non-expert musicians. Neuropsychologia, 108824.
Social media jurors conceptualizing and analyzing online public engagement in reference to legal cases
(Political candidates that admit to some criticisms may simultaneously attempt to link the opposition to perceived worse ones, e.g. both leading considered aged but showing different effects.)
Pan, C. A., Yakhmi, S., Iyer, T. P., Strasnick, E., Zhang, A. X., & Bernstein, M. S. (2022). Comparing the perceived legitimacy of content moderation processes: Contractors, algorithms, expert panels, and digital juries. Proceedings of the ACM on Human-Computer Interaction, 6(CSCW1), 1-31.
Digital juries.
Gordon, M. L., Lam, M. S., Park, J. S., Patel, K., Hancock, J., Hashimoto, T., & Bernstein, M. S. (2022, April). Jury learning: Integrating dissenting voices into machine learning models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (pp. 1-19).
(How is a juror instructed to eliminate implicit bias? What would be the specifics of a course that changed their minds? This is fairly easy to trigger in practice, e.g. as subtext to invoke irony.)