UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
Robust Preference Learning for Storytelling via Contrastive Reinforcement Learning
Metcalfe's law - Wikipedia
Metcalfe's law states that the value of a telecommunications network is proportional to the square of the number of connected users
of the system (n2). First formulated in this form by George Gilder in 1993,[1] and attributed to Robert Metcalfe in regard to Ethernet, Metcalfe's law was originally presented, c. 1980, not in terms of users, but rather of "compatible communicating devices" (e.g., fax machines, telephones).[2] Only later with the globalization of the Internet did this law carry over to users and networks as its original intent was to describe Ethernet connections.
Tokenization
Given a character sequence and a defined document unit, tokenization is
the task of chopping it up
into pieces, called tokens , perhaps
at the same time
throwing away certain characters, such as punctuation
Goodhart's law - Wikipedia
Goodhart's law is an adage often stated as, "When a measure becomes a target, it ceases to be a good measure".[1] It is named after British economist Charles Goodhart, who is credited with expressing the core idea of the adage in a 1975 article on monetary policy in the United Kingdom
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes
AI alignment - Wikipedia
In the field of artificial intelligence (AI), AI alignment research aims to steer AI systems towards their designers’ intended goals and interests.[a] An AI system is described as misaligned if it is competent but advances an unintended objective
Dynamic programming - Wikipedia
Dynamic programming is both a mathematical optimization method and a computer programming method. The method was developed by Richard Bellman in the 1950s and has found applications in numerous fields, from aerospace engineering to economics.
In both contexts it refers to simplifying a complicated problem by breaking it down into simpler sub-problems in a recursive manner
Non-governmental organization - Wikipedia
A non-governmental organization (NGO) or non-governmental organisation (see spelling differences) is an organization that generally is formed independent from government.[2][3][4][5][6] They are typically nonprofit entities, and many of them are active in humanitarianism or the social sciences; they can also include clubs and associations that provide services to their members and others. Surveys indicate that NGOs have a high degree of public trust, which can make them a useful proxy for the concerns of society and stakeholders.[7] However, NGOs can also be lobby groups for corporations, such as the World Economic Forum.
Secure multi-party computation - Wikipedia
Secure multi-party computation (also known as secure computation, multi-party computation (MPC) or privacy-preserving computation) is a subfield of cryptography with the goal of creating methods for parties to jointly compute a function over their inputs while keeping those inputs private. Unlike traditional cryptographic tasks, where cryptography assures security and integrity of communication or storage and the adversary is outside the system of participants (an eavesdropper on the sender and receiver), the cryptography in this model protects participants' privacy from each other.
automatic differentiation nutshell
Automatic differentiation - Wikipedia
Automatic differentiation is distinct from symbolic differentiation and numerical differentiation.
Symbolic differentiation faces the difficulty of converting a computer program into a single mathematical expression and can lead to inefficient code. Numerical differentiation (the method of finite differences) can introduce round-off errors in the discretization process and cancellation. Both of these classical methods have problems with calculating higher derivatives, where complexity and errors increase. Finally, both of these classical methods are slow at computing partial derivatives of a function with respect to many inputs, as is needed for gradient-based optimization algorithms. Automatic differentiation solves all of these problems.
NeuralNet from scratch
MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuning
Large-scale pretraining is fast becoming the norm in Vision-Language (VL)
modeling. However, prevailing VL approaches are limited by the requirement for
labeled data and the use of complex multi-step pretraining objectives. We
present MAGMA - a simple method for augmenting generative language models with
additional modalities using adapter-based finetuning. Building on Frozen, we
train a series of VL models that autoregressively generate text from arbitrary
combinations of visual and textual input. The pretraining is entirely
end-to-end using a single language modeling objective, simplifying optimization
compared to previous approaches. Importantly, the language model weights remain
unchanged during training, allowing for transfer of encyclopedic knowledge and
in-context learning abilities from language pretraining. MAGMA outperforms
Frozen on open-ended generative tasks, achieving state of the art results on
the OKVQA benchmark and competitive results on a range of other popular VL
benchmarks, while pretraining on 0.2% of the number of samples used to train
SimVLM.
MAGMA - a simple method for augmenting generative language models with
additional modalities using adapter-based finetuning. Building on Frozen, we
train a series of VL models that autoregressively generate text from arbitrary
combinations of visual and textual input. The pretraining is entirely
end-to-end using a single language modeling objective, simplifying optimization
compared to previous approaches. Importantly, the language model weights remain
unchanged during training, allowing for transfer of encyclopedic knowledge and
in-context learning abilities from language pretraining. MAGMA outperforms
Frozen on open-ended generative tasks, achieving state of the art results on
the OKVQA benchmark and competitive results on a range of other popular VL
benchmarks, while pretraining on 0.2% of the number of samples used to train
SimVLM.
Effect of Regularization in Neural Net Training
co-authored with Daryl Chang
On applying dropout, the distribution of weights across all layers changes from a zero mean uniform distribution to a zero mean gaussian distribution. This is similar to the weight decaying effect of L2 regularization on model weights
Linear separability: Sparse representations are also more likely to be linearly separable, or more easily separable with less non-linear machinery, simply because the information is represented in a high-dimensional space.
Rational choice theory - Wikipedia
Rational choice theory refers to a set of guidelines that help understand economic and social behaviour.[1] The theory originated in the eighteenth century and can be traced back to political economist and philosopher, Adam Smith.[2] The theory postulates that an individual will perform a cost-benefit analysis to determine whether an option is right for them.[3] It also suggests that an individual's self-driven rational actions will help better the overall economy. Rational choice theory looks at three concepts: rational actors, self interest and the invisible hand.
Power (social and political) - Wikipedia
In rational choice theory, human individuals or groups can be modelled as 'actors' who choose from a 'choice set' of possible actions in order to try to achieve desired outcomes. An actor's 'incentive structure' comprises (its beliefs about) the costs associated with different actions in the choice set, and the likelihoods that different actions will lead to desired outcomes.
In this setting we can differentiate between:
outcome power – the ability of an actor to bring about or help bring about outcomes;
social power – the ability of an actor to change the incentive structures of other actors in order to bring about outcomes.
This framework can be used to model a wide range of social interactions where actors have the ability to exert power over others. For example, a 'powerful' actor can take options away from another's choice set; can change the relative costs of actions; can change the likelihood that a given action will lead to a given outcome; or might simply change the other's beliefs about its incentive structure.
Emotional intelligence - Wikipedia
Emotional intelligence (EI) is most often defined as the ability to perceive, use, understand, manage, and handle emotions. People with high emotional intelligence can recognize their own emotions and those of others, use emotional information to guide thinking and behavior, discern between different feelings and label them appropriately, and adjust emotions to adapt to environments.[1] Although the term first appeared in 1964,[2] it gained popularity in the 1995 best-selling book Emotional Intelligence, written by science journalist Daniel Goleman. Goleman defined EI as the array of skills and characteristics that drive leadership performance
Strengths & Weaknesses | Architect (INTJ) Personality | 16Personalities
Socially Clueless – Architects’ relentless rationality can lead to frustration in their social lives. Their efforts to defy expectations may leave them feeling isolated or disconnected from other people. At times, they may become cynical about the value of relationships altogether, questioning the importance of love and connection.
Romantic Relationships | Architect (INTJ) Personality | 16Personalities
Architects (INTJs) approach romance the way they do most challenges: strategically, with clear-cut goals and a plan for achieving them. In a purely rational world, this approach would be foolproof. Alas, love is rarely rational, and Architects are at risk of overlooking or misinterpreting the unpredictability of human nature and affection.
Protein structure generation via folding diffusion
The misuse of colour in science communication
Ben Horowitz - Wikipedia
This transaction transferred 100% of Loudcloud's revenue to EDS while the company was publicly traded on NASDAQ. Beginning with EDS as its first enterprise software customer, Horowitz grew Opsware to hundreds of enterprise customers, over $100 million in annual revenue, and 550 employees. In July 2007, Horowitz sold Opsware to Hewlett-Packard for $1.6 billion in cash
Friedrich Nietzsche: On Love And Hate - Farnam Street
Nietzsche "He who promises to love forever or hate forever or be forever faithful to someone is promising something that is not in his power."
We must learn to love, learn to be kind, and this from earliest youth … Likewise, hatred must be learned and nurtured, if one wishes to become a proficient hater
Kindness and love, the most curative herbs and agents in human intercourse, are such precious finds that one would hope these balsamlike remedies would be used as economically as possible; but this is impossible. Only the boldest Utopians would dream of the economy of kindness
One can promise actions, but not feelings, for the latter are involuntary. He who promises to love forever or hate forever or be forever faithful to someone is promising something that is not in his power. He can, however, promise those actions that are usually the consequence of love, hatred, or faithfulness, but that can also spring from other motives: for there are several paths and motives to an action. A promise to love someone forever, then, means, ‘As long as I love you I will render unto you the actions of love; if I no longer love you, you will continue to receive the same actions from me, if for other motives.’ Thus the illusion remains in the minds of one’s fellow men that the love is unchanged and still the same.
One is promising that the semblance of love will endure, then, when without self-deception one vows everlasting love.
Bone Marrow Transplant: Types, Procedure & Risks
A bone marrow transplant is a medical procedure performed to replace bone marrow that has been damaged or destroyed by disease or chemotherapy.
Allogeneic transplants involve the use of cells from a donor. The donor must be a close genetic match. Often, a compatible relative is the best choice, but genetic matches can also be found from a donor registry.Allogeneic transplants are necessary if you have a condition that has damaged your bone marrow cells. However, they have a higher risk of certain complications, such as GVHD. You’ll also probably need to be put onmedications to suppress your immune system so that your body doesn’t attack the new cells. This can leave you susceptible to illness.The success of an allogeneic transplant depends on how closely the donor cells match your own.
Clinical trial - Wikipedia
Clinical trial costs vary depending on trial phase, type of trial, and disease studied. A study of clinical trials conducted in the United States from 2004 to 2012 found the average cost of Phase I trials to be between $1.4 million and $6.6 million, depending on the type of disease. Phase II trials ranged from $7 million to $20 million, and Phase III trials from $11 million to $53 million
Clinical trial - Wikipedia
Clinical trials are prospective biomedical or behavioral research studies on human participants designed to answer specific questions about biomedical or behavioral interventions, including new treatments (such as novel vaccines, drugs, dietary choices, dietary supplements, and medical devices) and known interventions that warrant further study and comparison
Non-communicable disease - Wikipedia
A non-communicable disease (NCD) is a disease that is not transmissible directly from one person to another. NCDs include Parkinson's disease, autoimmune diseases, strokes, most heart diseases, most cancers, diabetes, chronic kidney disease, osteoarthritis, osteoporosis, Alzheimer's disease, cataracts, and others. NCDs may be chronic or acute. Most are non-infectious, although there are some non-communicable infectious diseases, such as parasitic diseases in which the parasite's life cycle does not include direct host-to-host transmission.
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
Homomorphic encryption - Wikipedia
Homomorphic encryption is a form of encryption that permits users to perform computations on its encrypted data without first decrypting it. These resulting computations are left in an encrypted form which, when decrypted, result in an identical output to that produced had the operations been performed on the unencrypted data. Homomorphic encryption can be used for privacy-preserving outsourced storage and computation. This allows data to be encrypted and out-sourced to commercial cloud environments for processing, all while encrypted.
Dreamfusion