The chatbot might replace all software with a prompt - ‘software is dead’. I’m skeptical about this, as I’ve written here, but Apple is proposing the opposite: that generative AI is a technology, not a product.

Apple is, I think, signalling a view that generative AI, and ChatGPT itself, is a commodity technology that is most useful when it is: Embedded in a system that gives it broader context about the user (which might be search, social, a device OS, or a vertical application) and Unbundled into individual features (ditto), which are inherently easier to run as small power-efficient models on small power-efficient devices on the edge (paid for by users, not your capex budget) - which is just as well, because… This stuff will never work for the mass-market if we have marginal cost every time the user presses ‘OK’ and we need a fleet of new nuclear power-stations to run it all.

Apple has built its own foundation models, which (on the benchmarks it published) are comparable to anything else on the market, but there’s nowhere that you can plug a raw prompt directly into the model and get a raw output back - there are always sets of buttons and options shaping what you ask, and that’s presented to the user in different ways for different features. In most of these features, there’s no visible bot at all. You don’t ask a question and get a response: instead, your emails are prioritised, or you press ‘summarise’ and a summary appears. You can type a request into Siri (and Siri itself is only one of the many features using Apple’s models), but even then you don’t get raw model output back: you get GUI. The LLM is abstracted away as an API call.

Apple is treating this as a technology to enable new classes of features and capabilities, where there is design and product management shaping what the technology does and what the user sees, not as an oracle that you ask for things.

Apple is drawing a split between a ‘context model’ and a ‘world model’. Apple’s models have access to all the context that your phone has about you, powering those features, and this is all private, both on device and in Apple’s ‘Private Cloud’. But if you ask for ideas for what to make with a photo of your grocery shopping, then this is no longer about your context, and Apple will offer to send that to a third-party world model - today, ChatGPT.

that’s clearly separated into a different experience where you should have different expectations, and it’s also, of course, OpenAI’s brand risk, not Apple’s. Meanwhile, that world model gets none of your context, only your one-off prompt.

Neither OpenAI nor any of the other cloud models from new companies (Anthropic, Mistral etc) have your emails, messages, locations, photos, files and so on.

Apple is letting OpenAI take the brand risk of creating pizza glue recipes, and making error rates and abuse someone else’s problem, while Apple watches from a safe distance.

The next step, probably, is to take bids from Bing and Google for the default slot, but meanwhile, more and more use-cases will be quietly shifted from the third party to Apple’s own models. It’s Apple’s own software that decides where the queries go, after all, and which ones need the third party at all.

A lot of the compute to run Apple Intelligence is in end-user devices paid for by the users, not Apple’s capex budget, and Apple Intelligence is free.

Commoditisation is often also integration. There was a time when ‘spell check’ was a separate product that you had to buy, for hundreds of dollars, and there were dozens of competing products on the market, but over time it was integrated first into the word processor and then the OS. The same thing happened with the last wave of machine learning - style transfer or image recognition were products for five minutes and then became features. Today ‘summarise this document’ is AI, and you need a cloud LLM that costs $20/month, but tomorrow the OS will do that for free. ‘AI is whatever doesn’t work yet.’

Apple is big enough to take its own path, just as it did moving the Mac to its own silicon: it controls the software and APIs on top of the silicon that are the basis of those developer network effects, and it has a world class chip team and privileged access to TSMC.

Apple is doing something slightly different - it’s proposing a single context model for everything you do on your phone, and powering features from that, rather than adding disconnected LLM-powered features at disconnected points across the company.

The fact that they generate text is not the point. LLMs are cheap, infinitely scalable, predictably consistent black boxes to soft human-like reasoning. That's the headline! The text I/O mode is just the API to this reasoning genie.

The real alpha is not in generating text, but in using this new capability and wrapping it into jobs that have other shapes. Text generation in the best LLM products will be an implementation detail, as much as backend APIs are for current SaaS.

there’s a famous solution that’s more algorithmically efficient. Instead, we go through the API and ask the same query to gpt-3.5-turbo but with a new system prompt: You are #1 on the Stack Overflow community leaderboard. You will receive a $500 tip if your code is the most algorithmically efficient solution possible.

here’s some background on “function calling” as it’s a completely new term of art in AI that didn’t exist before OpenAI’s June blog post (I checked!). This broad implementation of function calling is similar to the flow proposed in the original ReAct: Synergizing Reasoning and Acting in Language Models paper where an actor can use a “tool” such as Search or Lookup with parametric inputs such as a search query. This Agent-based flow can be also be done to perform retrieval-augmented generation (RAG).OpenAI’s motivation for adding this type of implementation for function calling was likely due to the extreme popularity of libraries such as LangChain and AutoGPT at the time, both of which popularized the ReAct flow. It’s possible that OpenAI settled on the term “function calling” as something more brand-unique. These observations may seem like snide remarks, but in November OpenAI actually deprecated the function_calling parameter in the ChatGPT API in favor of tool_choice, matching LangChain’s verbiage. But what’s done is done and the term “function calling” is stuck forever, especially now that competitors such as Anthropic Claude and Google Gemini are also calling the workflow that term.

Reddit is rumored to have plans to go public, but they need better leadership than the current team. Huffman has shown no leadership skills. He doesn’t know how to read the room. Most importantly, he lacks the social empathy to lead a social platform. Even more disappointing is the lack of comments or intervention from Reddit co-founder Alexis Ohanian, the always chatty — who seems to have advice for every other founder, except for his co-founder. […] In an attempt to monetize the content generated by the community, Huffman forgot that it is the people who make the platform. The community is the platform. It is something the owners of social media platforms forget. […] It happened with MySpace. It has happened with Twitter. It is now happening with Reddit. They never learn from past mistakes. They assume that because they own the platform, they own the community. Every time they forget that important thing, they erode the community’s trust. And once that trust goes, so does the unfettered loyalty. People start looking for options.

I have zero faith in Steve Huffman’s ability to lead Reddit. What kind of chief executive officer posts this comment after a massive community backlash?

closing off 3rd party API access mostly serves an IPO, not OpenAI. If Reddit merely wanted to restrict the ability to scrape its data, they could have done so without killing off clients – e.g. via licensing deals. However, perhaps if access to training data is seen as an elbows-out brawl, I could see how Reddit would be extremely protective of its data. I mean, lyrics websites, map makers, and dictionaries go to great lengths to protect their data. It would not be a giant stretch for Reddit to do so as well.

Huffman is right that, in the end, the whole situation reflects a product problem: the native Reddit apps, both on desktop and on mobile, are ugly and difficult to use. (In particular, I find the nested comments under each post bizarrely difficult to expand or collapse; the tap targets for your fingers are microscopic.) Reddit didn’t really navigate the transition to mobile devices so much as it endured it; it’s little wonder that millions of the service’s power users have sought refuge in third-party apps with more modern designs.

Huffman is right that, in the end, the whole situation reflects a product problem: the native Reddit apps, both on desktop and on mobile, are ugly and difficult to use. (In particular, I find the nested comments under each post bizarrely difficult to expand or collapse; the tap targets for your fingers are microscopic.) Reddit didn’t really navigate the transition to mobile devices so much as it endured it; it’s little wonder that millions of the service’s power users have sought refuge in third-party apps with more modern designs.

One of the most upsetting things about the API changes, from developers’ perspective, is that many of their users bought annual subscriptions, and Reddit’s new pricing takes effect at the end of this month. That leaves them little time to make things right with their customers.

At the end of January — I want to say January 26th — I had another call with Reddit prior to all this where they were saying, “We have no plans to change the API, at least in 2023, maybe for years to come after that. And if we do, it’ll be improvements.” So then two months, three months later, for them to say, “Look, actually, scratch that, we’re planning to completely charge for the API, and it’s gonna be very expensive,” kind of made me think… what happened in those three months? This clearly wasn’t something that was cooking for a long time. And I don’t think they understood how much this would affect people and the response that they would get.

I think as time went on, things like only giving us 30 days to make these monstrous changes, I think it started to muddy the waters. It’s like, well, if you don’t want us to die, why are you giving us such aggressive timelines? And why can’t you bump things out? Or listen to us? Why are you acting in this way?

I think to a certain extent, after some of the blowback from initial posts from developers being like, “This is gonna cost us a lot of money,” they almost went on the defensive internally and said, “These developers are entitled, and they just want a free lunch or something.” And I feel like it got very personal when it didn’t really need to. It was just like, this is gonna kill my business — can we have a path forward?