Twitter has always been vague when it comes to limits in the API. The search API is a good example. Until version 1.1 the search API had rate limits, but they were never revealed in the docs or by dev support staff. The most we were able to extract were statements about there being enough requests to handle the needs of most apps. Eventually the guesstimate of 200 search API requests per hour became the unwritten assumption. Other areas still left uncertain are the windows of time used to rate limit actions like posting tweets and DMS during the day. We are given daily limits, such as posting up to 1,000 tweets a day, but within the day there are secret limits based on unspecified time periods. The number of apps an account is able to create is another secret. I have some ideas on how the API dev community may be able to fill in these gaps in the future, but first I’d like to explore the psychology and corporate strategy driving this secrecy.
Twitter is under constant attack
You have to look at this from the point of view of Twitter’s own developers and sysadmins. They are under attack from thousands, perhaps millions, of spammers and bots. There is @mention spam, reply spam, DM spam, follow spam, account creation spam, and probably kinds of spam I’ve never noticed. If you had to keep a massive server farm alive while getting beaten on constantly by waves of bots, you would naturally develop a bunker mentality. These are not true denial of service attacks, but they must feel like that when you are on the receiving end.
A classic way of dealing with online attacks is to reveal as few details as possible about your true limits. If you told everyone that they could only send X tweets per hour, spammers would instantly adjust their code to match X. Why make it easy for your attackers to just slip within your limits?
Unfortunately, before API 1.1 allowed Twitter to track every request to a specific account, it was impossible to tell the good devs from the spammers. I know that Twitter wants good developers to build on the API. If they didn’t it would just get turned off. But the primary task has been keeping the service going, and some good devs and their apps are just going to be collateral damage.
The Twitter API is a byproduct, not a product
The true role of the Twitter API is to support Twitter.com and Twitter’s own apps. Allowing outsiders to also access the API was a very clever hack by Twitter that started before APIs were assumed to be offered by web services. If the primary job of the API is driving Twitter’s own products, then the limits have to be flexible and adjust to a constantly changing feature set. Twitter can’t tie itself down by promising 3rd party developers that there will be an exact set of limits for every action. With the rate of change Twitter’s code undergoes I’m willing to bet that the support staff doesn’t even know what the limits are for many requests at a given time.
API limits have to react to demand and server load
For much of Twitter’s early years just keeping the service up in the face of rapid growth was a huge challenge. One obvious solution was to throttle back API limits during periods of heavy load. Scaling Twitter and managing the load has been largely solved, but when there is a revolution, or natural disaster, or someone like Mitt Romney makes a speech, the load still spikes. I’m sure the API is still throttled back at these times.
Thou shalt not reverse engineer
Finally we come to the lawyers. When reading Twitter’s terms of service it becomes clear that the lawyers were given the task of cutting off the possibility of a competitor duplicating Twitter. Since lawyers have to work with the constraints of the legal system and their own lack of technical knowledge, the common legal answer is “keep it vague.” Don’t reveal anything that will help a competitor understand your product’s internal machinery. I’ve dealt with IP lawyers enough to have gotten that answer many times. So management asks the lawyers what to say about limits, the lawyers say to reveal as little as possible, and this answer gets communicated back to the support staff.
What secret limits have you figured out? Do you know the true limits on search results returned or the number of apps an account can create? Share them here or tweet me.
