PANGU-Σ: TOWARDS TRILLION PARAMETER LANGUAGE MODEL WITH SPARSE HETEROGENEOUS COMPUTING
Public
REAC T: SYNERGIZING REASONING AND ACTING IN LANGUAGE MODELS
Electron gun - Wikipedia
An electron gun is an electrical component in some vacuum tubes that produces a narrow, collimated electron beam that has a precise kinetic energy.
An electron gun (also called electron emitter) is an electrical component in some vacuum tubes that produces a narrow, collimated electron beam that has a precise kinetic energy.
The largest use is in cathode-ray tubes (CRTs), used in nearly all television sets, computer displays and oscilloscopes that are not flat-panel displays. They are also used in field-emission displays (FEDs), which are essentially flat-panel displays made out of rows of extremely small cathode-ray tubes. They are also used in microwave linear beam vacuum tubes such as klystrons, inductive output tubes, travelling wave tubes, and gyrotrons, as well as in scientific instruments such as electron microscopes and particle accelerators.
Electron microscope - Wikipedia
An electron microscope is a microscope that uses a beam of accelerated electrons as a source of illumination. As the wavelength of an electron can be up to 100,000 times shorter than that of visible light photons, electron microscopes have a higher resolving power than light microscopes and can reveal the structure of smaller objects. A scanning transmission electron microscope has achieved better than 50 pm resolution in annular dark-field imaging mode and magnifications of up to about 10,000,000× whereas most light microscopes are limited by diffraction to about 200 nm resolution and useful magnifications below 2000×.
An electron microscope is a microscope that uses a beam of accelerated electrons as a source of illumination. As the wavelength of an electron can be up to 100,000 times shorter than that of visible light photons, electron microscopes have a higher resolving power than light microscopes and can reveal the structure of smaller objects. A scanning transmission electron microscope has achieved better than 50 pm resolution in annular dark-field imaging mode[1] and magnifications of up to about 10,000,000× whereas most light microscopes are limited by diffraction to about 200 nm resolution and useful magnifications below 2000×.
Air gap (networking) - Wikipedia
An air gap, air wall, air gapping[1] or disconnected network is a network security measure employed on one or more computers to ensure that a secure computer network is physically isolated from unsecured networks, such as the public Internet or an unsecured local area network.[2] It means a computer or network has no network interface controllers connected to other networks,[3][4] with a physical or conceptual air gap, analogous to the air gap used in plumbing to maintain water quality.
An air gap, air wall, air gapping[1] or disconnected network is a network security measure employed on one or more computers to ensure that a secure computer network is physically isolated from unsecured networks, such as the public Internet or an unsecured local area network.[2] It means a computer or network has no network interface controllers connected to other networks,[3][4] with a physical or conceptual air gap, analogous to the air gap used in plumbing to maintain water quality.
SELF-INSTRUCT: Aligning Language Model with Self Generated Instructions
Joint Embedding Methods - Contrastive · Deep Learning
Joint Embedding methods try to make their backbone network robust to certain distortions and are invariant to data augmentation.
Shor's algorithm - Wikipedia
If a quantum computer with a sufficient number of qubits could operate without succumbing to quantum noise and other quantum-decoherence phenomena, then Shor's algorithm could be used to break public-key cryptography schemes, such as
The RSA scheme
The Finite Field Diffie-Hellman key exchange
The Elliptic Curve Diffie-Hellman key exchange
Shor's algorithm is a quantum computer algorithm for finding the prime factors of an integer. It was developed in 1994 by the American mathematician Peter Shor
Transitioning organizations to post-quantum cryptography - Nature
Standards and recommendations for transitioning organizations to quantum-secure cryptographic protocols are outlined, including a discussion of transition timelines and the leading strategies to protect systems against quantum attacks.
The Model That Changes Everything: Alpaca Breakthrough (ft. Apple's LLM, BritGPT, Ernie and AlexaTM)
8 years of cost reduction in 5 weeks: how Stanford's Alpaca model changes everything, including the economics of OpenAI and GPT 4. The breakthrough, using self-instruct, has big implications for Apple's secret large language model, Baidu's ErnieBot, Amazon's attempts and even governmental efforts, like the newly announced BritGPT.
I will go through how Stanford put the model together, why it costs so little, and demonstrate in action versus Chatgpt and GPT 4. And what are the implications of short-circuiting human annotation like this? With analysis of a tweet by Eliezer Yudkowsky, I delve into the workings of the model and the questions it rises.
Web Demo: https://alpaca-ai0.ngrok.io/
Alpaca: https://crfm.stanford.edu/2023/03/13/alpaca.html
Ark Forecast: https://research.ark-invest.com/hubfs/1_Download_Files_ARK-Invest/Big_Ideas/ARK%20Invest_013123_Presentation_Big%20Ideas%202023_Final.pdf
Eliezer Tweet: https://twitter.com/ESYudkowsky/status/1635577836525469697
https://twitter.com/ESYudkowsky/status/1635667349792780288
Self-Instruct: https://arxiv.org/pdf/2212.10560.pdf
InstructGPT: https://openai.com/research/instruction-following
OpenAI Terms: https://openai.com/policies/terms-of-use
MMLU Test: https://arxiv.org/pdf/2009.03300.pdf
Apple LLM: https://www.nytimes.com/2023/03/15/technology/siri-alexa-google-assistant-artificial-intelligence.html
GPT 4 API: https://openai.com/pricing
Llama Models: https://arxiv.org/pdf/2302.13971.pdf
BritGPT: https://www.theguardian.com/technology/2023/mar/15/uk-to-invest-900m-in-supercomputer-in-bid-to-build-own-britgpt
Amazon: https://www.businessinsider.com/amazons-ceo-andy-jassy-on-chat-cpt-ai-2023-2?r=US&IR=T
AlexaTM: https://arxiv.org/pdf/2208.01448.pdf
Baidu Ernie: https://www.nytimes.com/2023/03/16/world/asia/china-baidu-chatgpt-ernie.html
PaLM API: https://developers.googleblog.com/2023/03/announcing-palm-api-and-makersuite.html
https://www.patreon.com/AIExplained
Multimodal MR-imaging reveals large-scale structural and functional connectivity changes in profound early blindness
In the setting of profound ocular blindness, numerous lines of evidence demonstrate the existence of dramatic anatomical and functional changes within the brain. However, previous studies based on a variety of distinct measures have often provided inconsistent findings. To help reconcile this issue, we used a multimodal magnetic resonance (MR)-based imaging approach to provide complementary structural and functional information regarding this neuroplastic reorganization. This included gray matter structural morphometry, high angular resolution diffusion imaging (HARDI) of white matter connectivity and integrity, and resting state functional connectivity MRI (rsfcMRI) analysis. When comparing the brains of early blind individuals to sighted controls, we found evidence of co-occurring decreases in cortical volume and cortical thickness within visual processing areas of the occipital and temporal cortices respectively. Increases in cortical volume in the early blind were evident within regions of parietal cortex. Investigating white matter connections using HARDI revealed patterns of increased and decreased connectivity when comparing both groups. In the blind, increased white matter connectivity (indexed by increased fiber number) was predominantly left-lateralized, including between frontal and temporal areas implicated with language processing. Decreases in structural connectivity were evident involving frontal and somatosensory regions as well as between occipital and cingulate cortices. Differences in white matter integrity (as indexed by quantitative anisotropy, or QA) were also in general agreement with observed pattern changes in the number of white matter fibers. Analysis of resting state sequences showed evidence of both increased and decreased functional connectivity in the blind compared to sighted controls. Specifically, increased connectivity was evident between temporal and inferior frontal areas. Decreases in functional connectivity were observed between occipital and frontal and somatosensory-motor areas and between temporal (mainly fusiform and parahippocampus) and parietal, frontal, and other temporal areas. Correlations in white matter connectivity and functional connectivity observed between early blind and sighted controls showed an overall high degree of association. However, comparing the relative changes in white matter and functional connectivity between early blind and sighted controls did not show a significant correlation. In summary, these findings provide complimentary evidence, as well as highlight potential contradictions, regarding the nature of regional and large scale neuroplastic reorganization resulting from early onset blindness.
If indeed adaptive behaviors observed in the blind are intimately related to changes in the overall structural and functional organization of the brain, evidence of increased morphological changes (e.g. gray matter volume or structural hypertrophy) and connectivity (white matter projections and functional connectivity) may be indicative of enhanced organization and facilitation of information processing occurring locally and/or between remote brain regions
The Four Types of Relationships
This article describes the four types of relationships and why only one of them sets you up for long term success as well as the reputational cue ball.
The single most important principle in biology is sustainability. In fact, it’s so important, that it means everything. Nothing matters if you can’t pass the test of time.
When human relationships are sustainable not only do they survive, they compound.
GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
Couairon embedding arithmetic of multimodal queries for image retrieval cvprw 2022 paper
Inflection point - Wikipedia
In differential calculus and differential geometry, an inflection point, point of inflection, flex, or inflection is a point on a smooth plane curve at which the curvature changes sign. In particular, in the case of the graph of a function, it is a point where the function changes from being concave to convex, or vice versa.
In differential calculus and differential geometry, an inflection point, point of inflection, flex, or inflection (British English: inflexion) is a point on a smooth plane curve at which the curvature changes sign. In particular, in the case of the graph of a function, it is a point where the function changes from being concave (concave downward) to convex (concave upward), or vice versa.
GPT-4 Technical Report
Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models
Training Compute-Optimal Large Language Models
Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations
Explained from scratch: private information retrieval using homomorphic encryption
Beyond k-anonymity: private password breach checking
LORA: LOW-RANK ADAPTATION OF LARGE LANGUAGE MODELS
Product quantization for nearest neighbor search
ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematics
A descriptive, not prescriptive, overview of current AI Alignment Research - LessWrong
TL;DR: In this project, we collected and cataloged AI alignment research literature and analyzed the resulting dataset in an unbiased way to identify major research directions. We found that the fiel…
LLaMA: Open and Efficient Foundation Language Models
Introducing LLaMA: A foundational, 65-billion-parameter language model
Today, we’re releasing our LLaMA (Large Language Model Meta AI) foundational model with a gated release. LLaMA is more efficient and competitive with previously published models of a similar size on existing benchmarks.
fka/awesome-chatgpt-prompts · Datasets at Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
World’s first on-device demonstration of Stable Diffusion on an Android phone
Qualcomm AI Research deploys a popular 1B+ parameter foundation model on an edge device through full-stack AI optimization.
Dynamic random-access memory - Wikipedia
Dynamic random-access memory is a type of random-access semiconductor memory that stores each bit of data in a memory cell, usually consisting of a tiny capacitor and a transistor, both typically based on metal–oxide–semiconductor (MOS) technology. While most DRAM memory cell designs use a capacitor and transistor, some only use two transistors. In the designs where a capacitor is used, the capacitor can either be charged or discharged; these two states are taken to represent the two values of a bit, conventionally called 0 and 1. The electric charge on the capacitors gradually leaks away; without intervention the data on the capacitor would soon be lost. To prevent this, DRAM requires an external memory refresh circuit which periodically rewrites the data in the capacitors, restoring them to their original charge. This refresh process is the defining characteristic of dynamic random-access memory, in contrast to static random-access memory (SRAM) which does not require data to be refreshed. Unlike flash memory, DRAM is volatile memory, since it loses its data quickly when power is removed. However, DRAM does exhibit limited data remanence.
Dynamic random-access memory (dynamic RAM or DRAM) is a type of random-access semiconductor memory that stores each bit of data in a memory cell, usually consisting of a tiny capacitor and a transistor, both typically based on metal–oxide–semiconductor (MOS) technology. While most DRAM memory cell designs use a capacitor and transistor, some only use two transistors