The Waiting Problem: A Closer Look at a Programmer's Productivity

6 min read

We're constantly told that Large Language Models (LLMs) are revolutionizing productivity in software development. The headlines promise dramatic efficiency gains, faster development cycles, and superhuman coding capabilities. Yet, if you're like me and use these tools daily, you've probably found yourself staring at a loading spinner more often than you'd care to admit, watching precious seconds tick by as you wait for a response.

This disconnect between the promised productivity revolution and the reality of our daily experience raises an important question: Are we actually becoming more productive, or are we just trading one set of inefficiencies for another?

The Productivity Paradox

The Promise vs. The Reality

The marketing narrative around LLMs is compelling:

  • "10x faster development"
  • "Reduce coding time by 80%"
  • "From idea to implementation in minutes"

But the day-to-day reality often looks quite different:

  • Waiting 30-60 seconds for a response to a complex query
  • Multiple iterations to get the right output
  • Time spent reviewing and debugging AI-generated code
  • Context switching while waiting for responses

The Hidden Costs of Waiting

Let's break down what actually happens during a typical AI-assisted coding session:

  1. Formulate the question (30 seconds)
  2. Wait for response (45 seconds)
  3. Review the output (60 seconds)
  4. Realize it's not quite right (15 seconds)
  5. Refine the prompt (45 seconds)
  6. Wait again (45 seconds)
  7. Finally get usable output (30 seconds)

Total time: 4 minutes and 30 seconds for what might have been a 2-minute task if done manually.

The Psychology of Waiting

Context Switching Costs

The most underestimated aspect of LLM usage isn't the wait time itself—it's what happens to our mental state during those waits. When we submit a query and wait for a response, we often:

  • Check email or messages
  • Start thinking about other problems
  • Lose the mental context we had built up
  • Experience what psychologists call "attention residue"

Research shows that it can take up to 23 minutes to fully refocus after an interruption. Even a 30-second wait can disrupt our flow state if we're not careful about how we handle it.

The Patience Paradox

There's an interesting psychological phenomenon at play: the longer we wait for something, the more valuable we perceive it to be. This "effort justification" bias can make us overvalue AI responses simply because we waited for them, leading to:

  • Less critical evaluation of AI output
  • Reluctance to iterate or ask for improvements
  • Acceptance of "good enough" solutions

Real-World Performance Analysis

Measuring Actual Productivity

To understand the true impact on productivity, we need to look beyond the immediate task completion time. Here's what I've observed in my own work:

Tasks Where LLMs Excel Despite Wait Times:

  • Documentation generation: Even with wait times, faster than writing from scratch
  • Boilerplate code: Wait time is justified by the volume of code generated
  • Complex explanations: Learning benefit outweighs the time cost

Tasks Where Wait Times Kill Productivity:

  • Simple debugging: Often faster to debug manually
  • Quick code snippets: Traditional IDE autocomplete is faster
  • Iterative refinement: Multiple round trips become time-consuming
  • Time-sensitive fixes: Can't afford to wait during production issues

The Compounding Effect

The productivity impact isn't just about individual queries—it's about the cumulative effect:

  • Daily wait time: 15-30 minutes of pure waiting
  • Context switches: 20-40 interruptions to flow state
  • Mental fatigue: Decision fatigue from constant AI output evaluation
  • Over-reliance: Reduced confidence in manual problem-solving

Strategies for Maximizing True Productivity

1. Strategic Query Batching

Instead of making individual requests:

  • How do I handle errors in this function?
  • What's the best way to log these errors?
  • How should I test error handling?

Try batching requests:

Implement comprehensive error handling for this function, including error catching, logging, and testing strategies.

2. Productive Waiting

Use wait times strategically:

  • Review previous code while waiting
  • Plan the next development step
  • Write comments or documentation
  • Organize imports or clean up formatting

3. Know When Not to Use AI

Develop intuition for when manual work is faster:

  • Simple syntax questions (use documentation)
  • Quick debugging (use IDE tools)
  • Familiar patterns (rely on experience)
  • Time-critical situations (prioritize speed)

4. Optimize Your Prompting

Better prompts reduce iteration cycles:

  • Include relevant context upfront
  • Specify format and style preferences
  • Mention constraints and requirements
  • Ask for explanations alongside code

The Infrastructure Reality

Response Times Aren't Getting Better Fast Enough

Despite improvements in model efficiency, response times remain a significant factor because:

  • Model complexity is increasing: More sophisticated models are slower
  • Usage is exploding: Server load affects response times
  • Context windows are growing: More data to process per request
  • Quality demands increase: We expect more from each response

The Economics of Waiting

From a business perspective, if a developer earning $100/hour spends 30 minutes daily waiting for AI responses, that's $50 in direct cost per day, not counting the indirect costs of context switching and reduced flow state.

Multiply this across a team of 20 developers, and you're looking at $1,000 daily, or over $240,000 annually in wait time costs alone.

Looking Forward: The Path to Real Productivity

What Needs to Improve

  1. Response Times: Sub-5-second responses for common queries
  2. Contextual Awareness: Less need for re-explanation
  3. Proactive Suggestions: AI that anticipates needs

What We Can Control Today

  1. Selective Usage: Use AI for high-value tasks where wait time is justified
  2. Workflow Optimization: Structure work to minimize wait-time disruption
  3. Skill Balance: Maintain manual coding skills for when AI isn't optimal
  4. Realistic Expectations: Understand that productivity gains are task-dependent

Conclusion

The promise of LLMs dramatically increasing developer productivity isn't wrong, it's incomplete. The technology genuinely enables us to tackle complex problems more effectively and handle certain types of work faster than ever before. However, the reality of wait times, context switching, and the overhead of human-AI collaboration means the productivity gains are more nuanced than the headlines suggest.

True productivity in the age of AI isn't about blindly using these tools for everything. It's about understanding when and how to use them effectively, managing the inherent trade-offs, and building workflows that account for the realities of current technology limitations.

As we wait for the next breakthrough in response times and model efficiency, the most productive developers will be those who master the art of strategic AI usage, knowing when to leverage these powerful tools and when to rely on their own expertise and the immediacy of traditional development approaches.

The future of AI-assisted development isn't about eliminating wait times, it's about making every second of wait time worth it through better tools, smarter usage patterns, and more realistic expectations about what productivity really means in this new era of development.

Follow me on X