Rate limiting advice: what I have learned

[cyconx]

🏷️ Discussion Type

Bug

💬 Feature/Topic Area

Copilot in GitHub

Body

Background
We are a small startup. We are working hard to complete a major release and have developed a very streamlined and mostly automated product pipeline.

From ideation, through architectural governance, design, planning, spikes, development, 10 layers of different types of testing, CI/CD, and production support almost every step is automated and cloud based. Fitting in AI as a sparring partner to help analyse code for specific patterns, review ideas, and so on was very natural to us. It quickly became a critical component of the teams overall productivity.

We made a big mistake
We have built in a lot of resilience in the customer facing product. But we haven't built in resilience in our own pipeline. The rate limit issue has reminded us of what we knew already. If you have an important process it must be resilient. There are many well understood patterns which can be relied on to resolve this challenge.

The Ugly Truth
If you are a power user and need AI assistance as part of your development process then make sure you have built in enough resiliency in your access to this resource. GitHub Copilot is great, it is why we are all so upset it has stopped working. But it is not the only tool in town.

GitHub can choose its customers, just like my business is choosing its customers by choosing which features to prioritise, setting prices, investment decision in infrastructure, rate limiting rules and so on. GitHub has chosen they do not want the power user customer outside the enterprise envelope. That is a fact.

What can you do about it
Think like Polly (may only make sense to C# developers). Design your consumption of AI assistance with resilience in mind. Do not rely on only one provider, ensure if one service isn't working you can switch without losing productivity. And you can add whatever resilience strategy makes sense to you. GitHub Copilot is great, but do not rely on it as you only service provider. And if you are a heavy user, do not rely on it as your main service provider.

Switching from one trusted provider to a combination of many (more than one), and rebuilding a pipeline that has worked for you so far is hard, but as engineers we already know how resilience works and the long term value it will bring.

Final take away
Just like the girl said to me in high school: "It is not me that is the problem, it is you".

Power Users of GitHub Copilot are not the target market, you have to move on.

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[Cyrus-DropPrint]

Your post makes little sense. You say GitHub can choose its customers like your business, but my understanding is when you have people pay for a service, and agree to a legally binding contract, that if you take away the service halfway through the billing period, that is called a SCAM, not called "choosing your customers". If they changed it and gave warning, that would be understandable, but just disabling it for days at a time, mid cycle, no warning, is not.

You also say "GitHub has chosen they do not want the power user customer outside the enterprise envelope. That is a fact.", where did you get this from? Do enterprise customers have lower rate limits?

[cyconx]

I am not saying I am condoning what GitHub is doing. As a matter of fact, you will find many posts from me shouting about the extremely poor service provided by GitHub in this instance. Let's be honest: GitHub's roll out of the changes to rate limits for Pro+ power users has been absolutely TERRIBLE 💩.

What I am saying: vote with your feet. Think like an engineer and protect yourself from dependency failures outside your control.

(My point about Enterprise customers is this: If they were impacted then they would have voiced their concerns as well. I only see critical comments from Pro+ users. SO I am guessing, but it is a somewhat informed guess🤔)

[Cyrus-DropPrint]

Gotchu, thanks. What tools would you recommend aside from Claude Code which I have found to not meet my needs (when I have used it with Opus 4.6, it was 37USD in API costs for 2 messages, and it only wrote 1/4 the code it should have, while GitHub copilot was able to easily do it for 0.12c before the rate limits).

[cyconx]

Gosh, only just started down that road myself. I will try to record my experience below. But it will be a combination of: googling, reading reviews, videos, plus trial & error.

So far Claude Code in VS Code is working out fine for me. I will still keep GitHub Copilot in full fat Visual Studio -- but only for one-off and IDE based work. We will also looking at using other options, but I am reluctant for us to host anything ourselves, so we will need to find AI model API providers or some other mechanism.

The painful truth is that we are at the mercy of Claude Code Opus 4.6/4.7 at the moment because that it the only model that currently will perform complex analysis or development work on our code base. But, the target is for us to have at least three different ways to access Claude.

We are also deliberately considering which model is sufficient for which task. Before we just used the same model every time regardless of the task.

[cyconx]

One way GitHub could have avoided a lot of problems would be to provide some visibility. From VS Code running Claude Code using Opus 4.7

[cyconx]

Another benefit of using more than one provider is that you can learn different things from different tools. I have only just realised that the way my copilot-instructions is defined is hammering context. It is obvious when you think about it, but because of the poor visibility provided by GitHub Copilot it is not something you notice.

I am now splitting my claude.md file into assembly specific files where it makes sense.

[cyconx]

Also, restart your session often. It will reduce the context significantly. I suspect a large context is one of the main reasons for rate limiting.