Board

Board

2272 bookmarks
Custom sorting
LAION, The Pile, and more datasets
LAION, The Pile, and more datasets
What's actually used to train these LLMs? A brief look at some of the datasets involved. LAION-5B Stable Diffusion was trained on a dataset called LAION-5B ("Large-scale Artificial Intelligence Open Network"), which is comprised of 5.85 billion image-text pairs crawled from the internet. The actual crawled data comes from Common Crawl. Common Crawl 3.15 billion pages contained in 380 TB. OpenAI's GPT-3 was, in part, trained by the data in Common Crawl. It is a non-profit founded by Gil Elbaz
·matt-rickard.com·
LAION, The Pile, and more datasets
Google I/O and the Coming AI Battles
Google I/O and the Coming AI Battles
Google A/I suggests that AI is a sustaining innovation for all of Big Tech; that means the real battle will be between incumbents and Big Tech on one side, and open source on the other.
·stratechery.com·
Google I/O and the Coming AI Battles
Accel - 40 Years in Tech
Accel - 40 Years in Tech
Technology has transformed our world and paved the way for a bright future. Join us as we reflect on key moments of innovation since 1983.
·40-years.accel.com·
Accel - 40 Years in Tech
User-Defined Functions in Databases vs SaaS - what's the difference?
User-Defined Functions in Databases vs SaaS - what's the difference?
Co-authored by Carl Sverre from SingleStore What is a User-Defined Function? A User-Defined Function (UDF) encapsulates business logic in such a way that it can be safely run within another service's infrastructure. Historically, general purpose UDFs...
When reaching for a UDF system as a tool to solve a problem, ask yourself this: “am I modifying my own system’s behaviour, or is someone else modifying my system?”. If the answer is that you want to modify the behaviour of your own system, a database UDF may be a good option. If you want to grant someone else the ability to modify how your software behaves, then a SaaS UDF is the way to go. Permalink
·blog.suborbital.dev·
User-Defined Functions in Databases vs SaaS - what's the difference?
To Understand Pants, Understand Bazel’s History
To Understand Pants, Understand Bazel’s History
Unlike I think pretty much all other professionals that don’t have it within their power to create their own tools – if you work in sales and you want better sales tools, you have to find a software engineer to do it for you – but we are software engineers and the tools that we use are themselves made out of software. So we have it in our power to fix them.
[I] went to work at Foursquare, I quickly noticed that Foursquare had the exact same problem. They had this big Scala code base and it wasn’t scaling. The solution at the time – and I am not joking – was to give all of the engineers a stick of ram, a screwdriver, and to say just upgrade your laptops.
I work on Earthly, another open-source build tool tackling similar problems. To Benjy, though, the important thing is not the potential for competition, it’s the size of the problem. I think an example of how much work there is to do in this space is the fact that Earthly and Pants are so different in their approaches, and yet both really fill in these needs.
There’s so much open space here to fill with good technology that two systems with radically different architectures and radically different approaches can both be very useful in their own right and also complimentary.
·earthly.dev·
To Understand Pants, Understand Bazel’s History
The Unending Chasm, and How to Survive It | Andreessen Horowitz
The Unending Chasm, and How to Survive It | Andreessen Horowitz
“Crossing the chasm” is a popular concept for almost all new products/startups, and is a useful lens for entrepreneurs to view the theory of innovation. The concept was first coined in the popular book title of the same name, which …
But once the chasm is crossed, hallelujah! — those product-market fit issues are solved and the challenge now is to just keep up with demand. Because once you cross the chasm, your company goes from a difficult “push-based market” to one that is “pull-based”, where customers are naturally drawn in.
Moreover, given how fast technology is changing, more startups are spending more time in the chasm… and they may never exit. And guess what? That’s ok.
All of the above means startups will face fierce and changing competition well into IPO territory, given the size of the markets relative to the size of a startup. But the whole point of this post is to argue that companies can still be successful despite never crossing the chasm, including IPO and beyond.
It’s simply a fallacy to believe that in all cases a market matures and starts to “pull” a product rather than requiring a continued “push” from the startup.
Whatever the reason, a product in these markets won’t be become repeatably “easy” to sell until well after it has reached hundreds of millions of dollars in sales. If you have such a product, that’s ok. In these situations, I generally recommend leaning into services. If the product is difficult to insert, you can reduce that friction by making the work to insert the product a core competency of your startup. While margins will be impacted, and sales cycles are likely to remain long, you’ll have more control of your destiny this way. If you’re lucky, over time as the market and partner ecosystem matures, you may be able to offload the integration work to a partner.
You know you’re in a hard, sometimes un-crossable market when it takes a lot of effort to build a business based on the ideal buyer for your product. In my experience, the two most common examples of these are: (1) non-“tech” verticals and (2) tech selling to struggling industries.
Every time I see a startup whose primary logos come from struggling sectors, I immediately recognize that they’re going to have a harder time going to market — their customer base is under duress. Even with a compelling product that has strong ROI for those customers, the churn of a buyer undergoing disruption will be reflected in the numbers. Budgets dry up, champions leave their jobs, projects are canceled, and so on. The hardest part for founders to to accept here is that all this can happen independent of how well a startup executes on its go to market in those markets.
It’s great if an enterprise startup manages to find product-market fit and ends up with a repeatable sales model in a large market, getting to a point where the market pulls them (and not the other way around). In that case, scaling fast is everything.
Why bother then? Because it is always possible to fight your way to success, to have a shot at building something great. Most enterprise companies are built brick by (often miserable) brick. As my former board member and now partner Ben Horowitz once said it best, “There is always another move”.
·a16z.com·
The Unending Chasm, and How to Survive It | Andreessen Horowitz
That underdog DNA
That underdog DNA
Jason just penned a beautiful, succinct ode to the underdogs. Go read it. It's funny how finding just the right word unlocks the perfect mental image. We've often thought of ourselves as being in the corner of the small business, but that was never quite right. There are many kinds of small businesses, not all of them thinking of thems...
·world.hey.com·
That underdog DNA
We stand with the Underdogs
We stand with the Underdogs
What do they got? A big team, lots of money, a strong brand, seemingly unlimited resources, panache, reputation, all that. They’re established. They’re your competitors. You want to look away, but you see them everywhere. Their ads on your social, their name in the media, your dream clients on their website. But you know what else they...
·world.hey.com·
We stand with the Underdogs
The AI Startup Litmus Test
The AI Startup Litmus Test
Differentiation is critical for Generative AI startups. Use the AI Litmus Test to determine if your company is unique, hard and defensible.
·nfx.com·
The AI Startup Litmus Test
The New AI Moats
The New AI Moats
“We Have No Moat, And Neither Does OpenAI,” a supposedly leaked document from Google, makes some interesting points. The competitive landscape shifts, and so do the moats. What is no longer a moat Data is no longer a moat. For example, GPT-3 and Stable Diffusion were trained on public data sets by companies or groups with zero proprietary data. Now,
·blog.matt-rickard.com·
The New AI Moats
Second-level Thinking
Second-level Thinking
Howard Marks, the founder of Oaktree Capital Management, makes the distinction between first-level thinking and second-level thinking. First-level thinking is superficial analysis — investors (or any other decision makers) making decisions on market sentiment, recent news, or stock price. His examples:
·blog.matt-rickard.com·
Second-level Thinking
Self-hosted Compilers and Bootstrapped AI
Self-hosted Compilers and Bootstrapped AI
The Go compiler was initially written in C but is now entirely written in Go. The Rust compiler is written in Rust (initially in OCaml). These compilers are capable of compiling themselves. Linux is compiled and developed on Linux. PyPy is a self-hosted Python interpreter.
·blog.matt-rickard.com·
Self-hosted Compilers and Bootstrapped AI
llm.ts
llm.ts
There are over 100 different LLMs, with more shipping every day. They differ slightly in their architectures, and the data they were trained on, but all of them do text completion. It’s the APIs that are fragmented — OpenAI uses a “completions” endpoint with parameters like “
·blog.matt-rickard.com·
llm.ts
Unix Philosophy for AI
Unix Philosophy for AI
Text processing was the initial pitch for the development of Unix at Bell Labs (see An Oral History of Unix). It became more than that. Spell checkers in `ed` used the `sort` command. Then there was `AWK,` the text processing language used by the `awk` tool by Aho, Weinberger, and Kernighan. Then there were Unix pipes — the development that made the
·blog.matt-rickard.com·
Unix Philosophy for AI
The data view from AWS re:Invent
The data view from AWS re:Invent
A look at Data & Analytics announcements out of AWS re:Invent in Nov/Dec-22.
Last year's focus was on adding serverless flavors of their analytical & streaming engines, plus on expanding their data marketplace features. However, AWS's push towards "serverless" was soon criticized as only delivering auto-scaling capabilities rather than truly lifting infrastructural burdens off the customer. If it does not have on-demand pricing nor turn off fully when idle, it is not serverless by most cloud observers' definition (which is also AWS's own definition when they invented serverless functions with Lambda).
Beyond Redshift catching up to Snowflake on the separation of storage and compute, data sharing, and serverless with on-demand pricing, they have also caught up on data marketplace capabilities.
·hhhypergrowth.com·
The data view from AWS re:Invent
Dimension.dev pitch with 16 year old founder Tejas Ravishankar
Dimension.dev pitch with 16 year old founder Tejas Ravishankar
One of the most well put together pitches I've seen from anyone, much less a teenager in high school.Site: https://dimension.devContact: https://twitter.com/...
·youtube.com·
Dimension.dev pitch with 16 year old founder Tejas Ravishankar
The future of cloud development - Ampt
The future of cloud development - Ampt
We envision a radically different approach that makes building native apps in the cloud ridiculously easy.
·getampt.com·
The future of cloud development - Ampt