Sight reading is just one component of a big band drummer's performance. The other components are listening, knowledge of form and an awareness of the music you are playing. In other words, you wouldn't approach Glenn Miller's "In The Mood" the same way as you would an arrangement by Stan Kenton. Having played with both groups, let me tell you that there are major differences in style and approach that you can't get from reading the drum part.
Phrasing is impossible without good time. A firm but flexible foundation will cause the band to relax and swing. You have to remember that swing is a collective process, once the band feels it they will begin to breathe in the groove that you are laying down.
The code of big band playing is: Time first, Roadmap second, Figures third. It is imperative that you follow this code in a sight reading situation. Inexperienced drummers often play the code in reverse order and that's when trouble begins.
A compositional sense is necessary for any gig. Knowledge of a composer's or player's style will help you nail most any gig. How do you develop this? Listening to a lot of music is step one. Playing, not practicing, is step two.
There are many other elements of big band playing that I would like to cover and I hope that you will join in the conversation. All thoughts are welcome.