One comment I often get from users is that an archive is missing tweets.
While I am confident we have started stabilizing in this area especially with the performance improvements of the Streaming API, there are still cases where the Twitter API could have glitched or we have a small blip and something can be missed.
To proactively find any missed tweets we run a background process against every archive hashtag/keyword/person on file periodically scanning the Twitter Search API and comparing against our database.
If we find a tweet we already have, we dump it.
If we find one we don’t, we store it in our database.
However, this can be a time consuming process since we have so many archives to crawl though.
Therefore, as of today we have introduced a small little “Reset your archive” button on every archive page that will bump the priority on the archive so that it is crawled more quickly.
So if you see an archive missing tweets go ahead and press the button – and if you have any issues, contact us on our GetSatisfaction page.
Don’t forget though – if you can’t find the tweet in search.twitter.com, TwapperKeeper can’t find it either.
(NOTE: This enhancement aligns with one of the enhancements outlined in the https://twapperkeeper.wordpress.com/2010/04/27/planned-ops-infrastructure-enhancements/)