QInsights AI Compass

Dear

You’ve probably heard claims that generative AI can analyze vast amounts of data effortlessly. Unfortunately, that’s not entirely true—and that’s what I want to talk about in this newsletter.

The Context Window: The AI’s Short-Term Memory

Theoretically, generative AI could analyze vast amounts of qualitative data. However, there’s a crucial limitation that often goes unnoticed: the context window. Think of it as the AI’s short-term memory—it can only process a fixed amount of data at a time before it needs to "forget" and start over.

This limit exists for a reason. Expanding the context window isn’t just a technical challenge; it comes with exponential computational costs. Doubling the context window size doesn’t just double the computing power needed—it quadruples it. If AI models were to process unlimited amounts of data at once, even the largest data centers wouldn’t be able to keep up.

How Much Data Can genAI Process at Once?

The size of the context window depends on the AI model and whether you're using a pro version:

• ChatGPT-4 (Pro version): 128k tokens (~250-350 pages or 10-15 one-hour interviews)

• Claude Pro (Anthropic’s model): 200k tokens (~400-500 pages)

This is already a remarkable capability, allowing AI to process and retain more information than was possible even a year ago. But I often hear researchers hoping to analyze 150 documents, or projects with 200-300 interviews in one go.

While strategies exist to work with large datasets, generative AI doesn’t function like a search engine that can simultaneously retrieve information from thousands of pages. 

If you need to analyze thousands of documents or work with truly large-scale datasets, generative AI alone is not the solution. Instead, this is where traditional AI methods—designed for large-scale text analysis—become essential. These approaches have been around for over a decade and provide structured, systematic ways to process vast amounts of text, including:

• Natural Language Processing (NLP): Identifies named entities, parses sentence structure, and tags parts of speech.

• TF-IDF (Term Frequency-Inverse Document Frequency): Ranks key terms and clusters documents based on keyword importance.

• Topic Modeling: Detects hidden themes in large text collections.

• Clustering (K-means, Hierarchical): Groups similar documents for structured analysis.

These techniques scale efficiently and provide consistent, repeatable insights—something generative AI cannot do on its own. For large-scale qualitative research, a combination of traditional AI methods and generative AI for deeper interpretation could be a possible solution.

What Happens When Your Data Exceed the Context Window?

When using a chatbot like ChatGPT or similar AI tools that allow data uploads, three possible scenarios can occur:

1. Data beyond the limit is ignored. Worst case scenario: you don’t even realize that part of your dataset wasn’t analyzed.

2. AI starts hallucinating. If it can’t access all the data, it may fabricate details rather than acknowledge the missing information.

3. The AI assistant warns you. Ideally, the chatbot notifies you that the data exceeds its limit and asks you to refine your request.

Most chatbots today typically allow you to upload up to 10 documents at a time. This limitation serves as an implicit indication that there’s a cap on how much data can be analyzed in a single session. If you try to bypass the system by uploading 10 very large documents, one of two things will likely happen:

1. Hard Limits on File Size or Token Count

   • Some AI tools analyze only what fits within the context window, regardless of how many documents you upload. If 10 large files exceed the model's capacity (e.g., 1000 pages far exceeding a 128k token limit), the tool may:

     • Truncate the text, keeping only the first portion.

     • Randomly or selectively include sections based on predefined rules.

     • Reject the upload outright if it exceeds system constraints.

2. Pre-Processing Before Analysis

   • Some platforms don't analyze everything at once but instead pre-process the documents, summarizing them, indexing key parts, or selecting relevant sections based on your query. This makes it feel like you're analyzing a large dataset at once, but in reality, the model only works with a subset of the data at any given moment.

How Does QInsights Handle Large Amounts of Data?

One way to work with large datasets is through Retrieval-Augmented Generation (RAG)—the approach used by QInsights. When your selected documents exceed the AI’s context window, QInsights pre-processes the data based on your query, retrieves only the most relevant segments, and processes them selectively.

For example, if you select 30 interviews for analysis and ask about the benefits of XY described by respondents, QInsights first extracts key excerpts relevant to your query—what we call a smart search. Instead of sending all 30 interviews in full, which would likely exceed the context window and waste computational resources, smart search ensures that only the necessary data is processed efficiently, keeping the analysis within system limits while preserving relevance. This enables the analysis of larger projects. However, it also comes with some trade-offs:

• Less granularity: The AI provides broader overviews rather than detailed, nuanced insights.

• Emphasis on frequent themes: Subtle patterns or unique perspectives may be overlooked.

• Risk of overgeneralization: Responses might become too broad, making it harder to pinpoint distinct variations.

If you need deeper, more detailed insights, a more effective approach is to narrow your focus—working with smaller, targeted subsets of data for more precise, iterative analysis.

QInsights supports this by allowing you to add speaker and document variables, which can be used as filters to create meaningful subsets. For example, by selecting a specific group of interviews based on criteria like demographics, profession, or experience level, you can:

• Uncover deeper nuances—the AI can focus on subtle variations rather than just generating broad summaries.

• Identify clearer patterns—reducing data overload helps the AI provide sharper, more relevant insights.

By refining your dataset strategically, you ensure that the analysis remains focused, meaningful, and rich in detail.

Final Takeaway: Smarter, Not Bigger, is the Key to Effective AI Analysis

While generative AI is a powerful tool, it has clear limits when handling large-scale qualitative data. The context window determines how much information AI can process at once, and exceeding it can lead to missing data, hallucinations, or overgeneralized summaries.

For truly large projects, combining generative AI with traditional methods like topic modeling, clustering, and NLP ensures a more structured and scalable approach. Instead of trying to force AI to analyze everything at once, working iteratively with targeted subsets leads to deeper, more meaningful insights.

With QInsights, smart search and filtering options help you manage large datasets efficiently, ensuring that AI-driven analysis remains focused, relevant, and insightful—without overwhelming the system.

I hope you found this insight valuable. Wishing you a great rest of the week—until next time!

With best regards,

Susanne

PS: If you know colleagues who might benefit from QInsights, feel free to invite them to try it out and see how it can enhance their research. Exploring QInsights is completely free.

🔗 Registration link