Beyond Just Sounding Right: Expert-Level Thinking With AI​

Session Description

LLMs sound great—but how close can they get to being right? Can we get them to deliver real, useful insights instead of just confident-sounding guesses?

Session Recap

1. Brief Summary

The session explored the intricacies of artificial intelligence, particularly Large Language Models, moving beyond their perceived ability to merely "sound right" towards achieving genuine expert-level thought. It delved into the mechanisms behind LLMs, including their probabilistic nature and how this can lead to "hallucinations" or generic, improbable answers. The discussion highlighted the critical differences between how LLMs process information and how human experts reason, emphasizing the need for tools that can ground these models in truth by providing them with specific, proprietary data and expert-defined methods.

2. Outline

• Introduction to Large Language Models (LLMs): An overview of how LLMs process petabytes of data, compressing it into vectors representing relationships between words, and then reconstructing information from prompts.
• The AI Stack and Context Window: Explanation of the basic model data layer, neural networks for training, and the application layer where users interact via an "inference" or "intention mechanism." The concept of the "context window," including guard rails, system prompts, and queries, was introduced, along with the probabilistic, token-by-token generation of responses.
• Limitations of Current LLMs:
  • Inability to Create Improbable Insights: Unlike the "infinite monkeys" thought experiment, LLMs struggle to generate unique or improbable answers because they are based on probabilities, averaging common data, making deep insights less likely.
  • Confidence vs. Competence: LLMs are aligned to produce confident-sounding text, but this confidence does not equate to actual knowledge or competence, often resulting in generic, non-authoritative output.
  • Blackbox Reasoning: It's impossible to understand why an LLM produces a particular output, and they will "confabulate" or make up reasoning if asked.
  • Lack of Critical Thinking: LLMs lack the ability to question, evaluate, justify, or be aware of their own limitations, unlike human reasoning.
• Distinguishing Expert Reasoning from LLM Capabilities: Experts draw deep insights by reasoning across multiple documents, relying on years of inaccessible experience, specialized tacit methods, and the ability to justify their reasoning step-by-step—abilities LLMs currently lack.
• Current Techniques to Improve LLMs:
  • Chain of Thought: Providing step-by-step instructions within the context window to force accuracy, though still limited to the model's existing data.
  • Retrieval Augmented Generation (RAG): Presenting the model with specific data to answer a question, often combined with Chain of Thought.
  • Prompt Chaining: Linking a series of prompts together for more complex queries.
  • Agents: Allowing for observability into the model's reasoning process and, critically, repeatability for consistent analysis of documents.
  • Game Techniques: Iterative ranking of multiple outputs to select the best one, though still relying on the model's inherent data limitations.
• Grounding LLMs in Truth for Expert-Level Outcomes: The session detailed a method of providing LLMs with proprietary, ground-truth data and precise instructions on how to analyze and break it down, significantly reducing hallucinations and enabling expert reasoning on private datasets like scientific papers or patent portfolios.

3. Notable Quotes

• "Pros must do more than just simply report. It must question, evaluate, and justify itself. So, reasoning itself must be transparent, evidence-based, fair-minded, and more importantly, it has to be aware of its own limitations. And large language models have none of these features".
• "You don't want to have a probable answer. You want to have an answer which is something that that that wouldn't have been there before that is not in the data necessarily, right? You want unique answers and thus deep insights are improbable and but improbable does not equal impossible".
• "The one thing that that you need to do with a large language model is present your own information into it. People tend to ask it for information as though we're a search engine, but instead you need to actually add your data first and then ask it to reason over that".

4. Key Takeaways

• Understand LLM Limitations: Be aware that LLMs are probabilistic, can "hallucinate," and lack true critical thinking or the ability to generate improbable, unique insights on their own. Confidence in an LLM's output does not guarantee competence.
• Experts Possess Unique, Inaccessible Knowledge: Human experts leverage tacit methods, hard-won experience, and the ability to justify reasoning that LLMs cannot access or replicate from public data.
• Ground LLMs with Your Data: To achieve expert-level results and reduce hallucinations, provide LLMs with your own specific, proprietary, and ground-truth data for them to analyze and reason over, rather than treating them as a general search engine.
• Beyond Simple Prompting: Techniques like Chain of Thought, RAG, and especially Agents, offer ways to introduce more structure, observability, and repeatability to LLM interactions, moving beyond basic prompt-response models.
• Seek Organized Resources for Deeper Understanding: While comprehensive, organized resources on truly understanding LLMs are scarce, seeking out specialized technical books (like those from O'Reilly) or insights from "translators" in the field can be beneficial for deeper learning.

5. Resources Mentioned

• O'Reilly Books: Referenced as a source for technical books on the subject, though noted they are not for general understanding.
• Claude and OpenAI: Mentioned as platforms where users can upload documents to a "project area," though the process can still be random.
• 4x9: The speaker's company, which focuses on providing LLMs with ground truth and expert reasoning systems to ensure repeatable and reliable processes.

6. Action Items

• Proactively Provide Your Own Data: When interacting with LLMs, shift your approach from querying for general information to feeding the model with your specific data first before asking it to reason or generate insights. This is crucial for obtaining unique, improbable, and expert-level answers.
• Explore Advanced Prompting Techniques: Consider using methods like Chain of Thought, Retrieval Augmented Generation (RAG), and Agents to improve the accuracy, transparency, and repeatability of LLM outputs.