how to get more than 100 tweets from the twitter search or streaming api using Twython

Sunday, May 11, 2014

How can i use Twython to retrieve all tweets that mention for example "#Accounting" for say in New York?


I used the Twitter search API but i was only allowed to get 100 tweets.


I then tried to use the Twitter streaming filter API but wasn't able to narrow by geolocation and it seemed to take forever. I read that we should put all that tweets into a database and then do the aggregations there but i was wondering if there was any other way to do something quick.


Here is my code:



from twython import Twython

TWITTER_APP_KEY = 'XXXX'
TWITTER_APP_KEY_SECRET = 'XXXX'
TWITTER_ACCESS_TOKEN = 'XXXX'
TWITTER_ACCESS_TOKEN_SECRET = 'XXXX'

t = Twython(app_key=TWITTER_APP_KEY,
app_secret=TWITTER_APP_KEY_SECRET,
oauth_token=TWITTER_ACCESS_TOKEN,
oauth_token_secret=TWITTER_ACCESS_TOKEN_SECRET)

search = t.search(q='#Accounting',
geocode='-74,40,-73,41',
since_id='1',
max_id='504082008759488512'
)

tweets = search['statuses']
count=0
for tweet in tweets:
count+=1
print tweet['id_str'], '\n', tweet['text'], '\n\n\n'

print count


And here is my code for the Streaming API:



from twython import TwythonStreamer
APP_KEY = 'XXXX'
APP_SECRET = 'XXXX'
OAUTH_TOKEN = 'XXXX'
OAUTH_TOKEN_SECRET = 'XXXX'

class MyStreamer(TwythonStreamer):
tweets=[]
def on_success(self, data):
if 'text' in data:
tweet= data['text'].encode('utf-8')
tweets.append(tweet)
print tweet
def on_error(self, status_code, data):
print status_code
self.disconnect()

stream = MyStreamer(APP_KEY, APP_SECRET,
OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
x=stream.statuses.filter(track='#Accounting')
x.stream.statuses.filter(location='-74,40,-73,41')


Thanks







http://ift.tt/1jflIpx