This page contains half-baked ideas that I am interested in thinking/writing/talking about. Minimal due diligence.
(Pro tip: instantly eliminate awkward silences by popping any of these questions the next time you see me!)
When AI gets something wrong, it’s a hallucination - or is it? Some researchers prefer “confabulations” or “factual errors”; Scott Alexander argues a better framing is shameless guesses. How do the metaphors we use with AI shape our understanding of and interaction with these [tools / systems / beings]?
The definitional analogue to artificial intelligence is natural intelligence, which is predominantly represented by human cognition. Notwithstanding the difficulties in defining intelligence itself, let alone the artificial aspect, there is a tendency to anthropomorphize AI - it learns, remembers, reasons, forgets. When do biological analogies help us reason about AI, and when are they misleading? If we work backwards from the failure modes, what do we learn about where the biological analogy breaks?
Many people are saying that AI will never have good taste. No matter how smart it gets, there will always be a human element to save white collar and creative professions. But what do people actually mean by taste? Is it a kind of qualia that is, by definition, human-only - second only to soul and consciousness in nebularity? Or could it be something more mundane: a bundle of compressed expertise and pattern recognition that AI has no theoretical barrier against learning?
(Alternatively: AI will never have taste, because I am the sole arbiter of taste and I have infinite capacity for moving the goalposts.)
I dislike reading LLM-generated text in the wild despite having no issue using chatbots, coding tools, etc., with almost identical stylistic choices. Many people I’ve talked to feel the same. Why the gap? One instinct is that the prose itself is bad - flavorless and stylistically incoherent - but then why is it so easy to gloss over in LLM applications? My guess is that there is some violation of an implicit contract between author and reader in certain situations, and the stronger that implicit expectation, the greater the disgust when it is violated. If I wanted to get an LLM’s thoughts on the topic, I could go direct to the source. When I interact with someone personally or professionally, it is because I want their thoughts - not regurgitated LLM thoughts. But this implies that the situation would be just as discomforting if the person read whatever the LLM told them, internalized and understood it, and then wrote it anew for me - and I think much fewer people would be offended by that. So what is at issue here?
AI memory is the hot new-ish thing. But what is memory? The immense variety in techniques in the field hints that the term is too broad. Is it useful to have a bucket so wide it covers both model-managed markdown files on one end to RAG over specialized databases augmented by heavy programmatic harnesses on the other? Maybe we start by figuring out what we want ‘memory’ to enable - longer-horizon projects, self-improving loops, better customization (for ads? yuck)? Each implies a different system and lumping these all together probably creates more confusion than clarity.