A course on getting started with the Twitter API v2 for academic research

Overview

Getting started with the Twitter API v2 for academic research

banner

Welcome to this '101 course' on getting started with academic research using the Twitter API. The objective of this course is to help academic researchers learn how to get Twitter data using the new Twitter API v2.

By the end of this course, you will learn:

  • What the Twitter API is
  • How to apply for the Academic Research product track and what’s available in it
  • How to identify the endpoints to use for your use-case
  • How to get data from the Twitter API v2 using Python and R
  • How to write and build search queries

Who is this course for?

This is an introductory course (101), meant for anyone who is interested in getting started with the Twitter API v2 for research including

  • Academic Researchers
  • Students
  • Independent Researchers

Note: While undergraduate students & independent researchers do not qualify for the academic research product track (which provides ability to search for Tweets older than 7 days), they can still follow this course and use the standard product track and the code samples associated with it.

For most of this course, there are no prerequisites and anyone can follow along. Specifically for module 6 which is the labs, you need to know very basic coding in Python or R. If you want to first learn or review the syntax for these two languages, check out the appendix section. It provides links to introductory material on Python and R, along with instructions on how to install Python and R.

Who is this course not for?

This is designed like a 100-level course. If you already gained access to the Academic Research product track, and/or, you already know how to get data from the Twitter API v2 using Python or R, this course may feel too “introductory” for you.

How is this course structured

This course consists of 8 modules. Use this course as a complete start-to-finish lesson for getting started, or if you already know some of the basics, you can start off on one of the more advanced modules later on in the course.

  • Module 1: Learn what the Twitter API v2 is, and see examples of research done with it
  • Module 2: Learn how to apply for a Twitter developer account and how to choose the right product track for your project
  • Module 3: Learn how to decide what Twitter API endpoints to use, based on the data you need
  • Module 4: Learn how to get your keys and bearer token from the developer dashboard to start using the Twitter API
  • Module 5: Learn how to write search queries to get Tweets from the Twitter API
  • Module 6 Labs in Python and R to learn how to write code and use libraries and packages to get Twitter data
  • Module 7: Learn how to store Twitter data once you receive it, as well as data compliance and best practices
  • Module 8: See a summary of what we learned in this course and find links for important resources for future reference.

There is also an Appendix that contains additional information and a glossary of terms used throughout this course, so it is a good idea to keep it handy (maybe even open in a new tab) and reference it whenever you come across a new term in this course.

Assumptions

Tweets

Whenever we refer to getting ‘Tweets’ using the Twitter API, we refer to only those Tweets that are publicly available. The Twitter API does not provide Tweet information for Tweets that have been deleted, and does not provide Tweets from users who have made their Tweets private.

V2 Only

We will only be using the new Twitter API v2 and not the old API (v1.1). To learn more about the Twitter API v2, check out this technical overview of the Twitter API v2.

Let us start with module 1, that provides an introduction to Twitter API and examples of research with it.

Get Started

Comments
  • Tweets and retweets only

    Tweets and retweets only

    Is there a combination to extract only the tweets and retweets of an account?

    In this way the mentions are also obtained: twarc2 search @barripdmx --archive test.jsonl

    Best regards barri

    opened by barripdmx 5
  • Fix a minor typo

    Fix a minor typo

    The 2nd line of 2nd paragraph under the section "Operator availability and query length" contained a typo. (".n the Standard product track..." instead of "In the Standard product track...").

    Problem

    Explain the context and why you're making that change. What is the problem you're trying to solve? In some cases there is not a problem and this can be thought of being the motivation for your change.

    Solution

    Describe the modifications you've done.

    Result

    What will change as a result of your pull request? Note that sometimes this section is unnecessary because it is self-explanatory based on the solution.

    opened by moingshaikh 1
  • Fix indentation

    Fix indentation

    Problem

    Explain the context and why you're making that change. What is the problem you're trying to solve? In some cases there is not a problem and this can be thought of being the motivation for your change.

    Solution

    Describe the modifications you've done.

    Result

    What will change as a result of your pull request? Note that sometimes this section is unnecessary because it is self-explanatory based on the solution.

    opened by L3str4nge 1
  • Postman develop module 6d

    Postman develop module 6d

    Problem

    Modules were missing a Postman learning resource to support the Twitter API V2 public collection.

    Solution

    Created a Module 6d, containing Postman enablement content.

    Result

    Note - The Postman public workspace which is associated with this learning module (6d) requires review before approval. Please see folder 13 in the Postman public workspace. Does the "batch compliance" copy accomplish the task which correlates with learning task 13? If it does not, there is an unfinished request which I've added to the folder. I would need clarity on what datasets are being compared for the learning task to be achieved in order to flesh out the request I've added. Please advise if the copied folder "Batch Compliance" achieves the task or if I should flesh out the additional request.

    opened by rubysattar 1
  • I can't read the Jsonl file with my twitter data

    I can't read the Jsonl file with my twitter data

    One line summary of the issue here.

    Expected behavior

    As concisely as possible, describe the expected behavior.

    Actual behavior

    As concisely as possible, describe the observed behavior.

    Steps to reproduce the behavior

    Please list all relevant steps to reproduce the observed behavior.

    opened by frabrusa 3
  • Update 6c-labs-code-academic-r.md

    Update 6c-labs-code-academic-r.md

    Problem

    Getting a Warning in install.packages : package ‘academicTwitteR’ is not available for this version of R error when trying to install the package.

    Solution

    Fixing the case of TwitteR to twitteR solved the issue.

    Result

    Edit to README.md

    opened by katkasian 1
  • twarc command line examples?

    twarc command line examples?

    This isn't a bug as much as an observation. First, I should say thank you for the examples using twarc in this documentation! As one of the maintainers of the module it's really nice to see it getting used and promoted by Twitter.

    One of the things that twarc tries to do is provide users with a reliable command line tool for collecting Twitter data so that they can use whatever language or tools they want to do their analysis. I was wondering if it might make sense to also provide command line versions of the operations you have here? People can do many of the things you describe here with twarc without writing any code at all. Would a PR along these lines be helpful to see what it would look like?

    opened by edsu 1
Owner
@TwitterDev
Code samples from Twitter Developer Relations for common use cases including REST API and Tweet display
@TwitterDev
Some examples regarding how to use the Twitter APIs for academic research

Twitter Developer Platform: Using Twitter APIs for Academic Research All the scripts require a config.ini file in which the keys are put. There is a t

Federico Bianchi 6 Feb 13, 2022
A template that help you getting started with Pycord.

A Pycord Template with some example! Getting Started: Clone this repository using git clone https://github.com/AungS8430/pycord-template.git If you ha

null 2 Feb 10, 2022
Actively maintained, pure Python wrapper for the Twitter API. Supports both normal and streaming Twitter APIs.

Twython Twython is a Python library providing an easy way to access Twitter data. Supports Python 3. It's been battle tested by companies, educational

Ryan McGrath 1.9k Jan 2, 2023
twitter bot tha uses tweepy library class to connect to TWITTER API

TWITTER-BOT-tweepy- twitter bot that uses tweepy library class to connect to TWITTER API replies to mentions automatically and follows the tweet.autho

Muziwandile Nkomo 2 Jan 8, 2022
Quickly and efficiently delete your entire tweet history with the help of your Twitter archive without worrying about the pointless 3200 tweet limit imposed by Twitter.

Twitter Nuke Quickly and efficiently delete your entire tweet history with the help of your Twitter archive without worrying about the puny and pointl

Mayur Bhoi 73 Dec 12, 2022
Twitter bot that finds new friends in Twitter.

PythonTwitterBot Twitter Bot Thats Find New Friends pip install textblob pip install tweepy pip install googletrans check requirements.txt file Env

IbukiYoshida 4 Aug 11, 2021
A twitter multi-tool for OSINT on twitter accounts.

>TwitterCheckr A twitter multi-tool for OSINT on twitter accounts. Infomation TwitterCheckr also known as TCheckr is multi-tool for OSINT on twitter a

IRIS 16 Dec 23, 2022
Twitter-bot - A Simple Twitter bot with python

twitterbot To use this bot, You will require API Key and Access Key. Signup at h

Bentil Shadrack 8 Nov 18, 2022
Twitter-redesign - Twitter Redesign With Django

Twitter Redesign A project that tests Django and React knowledge through a twitt

Mark Jumba 1 Jun 1, 2022
Assassination API for getting random quotes from Assassination Classroom.

Assassination API Take advantage of what you have, while you have it. Quotes from Assassination Classroom Assassination classroom is one of best anime

Swanand Mulay 3 Jul 15, 2022
An API or getting Optifine VersionsList/Version/Download-URL.

Optifine-API An API for getting Optifine VersionsList/Versions/Download-URL. Table of contents Get Versions List Get Specify Versions Download Optifin

null 2 Dec 4, 2022
A telegram bot providing recon and research functions for bug bounty research

Bug Bounty Bot A telegram bot with commands to simplify bug bounty tasks Installation Use Road Map Installation BugBountyBot is open-source so you can

Tyler Butler 1 Oct 23, 2021
A Python API wrapper for the Twitter API!

PyTweet PyTweet is an api wrapper made for twitter using twitter's api version 2! Installation Windows py3 -m pip install PyTweet Linux python -m pip

TheFarGG 1 Nov 19, 2022
Python API Client for Twitter API v2

?? Python Client For Twitter API v2 ?? Why Twitter Stream ? Twitter-Stream.py a python API client for Twitter API v2 now supports FilteredStream, Samp

Twitivity 31 Nov 19, 2022
Github repository started notify 💕

Github repository started notify ??

null 4 Aug 6, 2022
You can share your Chegg account for answers using this bot with your friends without getting your account blocked/flagged

Chegg-Answer-Bot You can share your Chegg account for answers using this bot with your friends without getting your account blocked/flagged Reuirement

Ammey Saini 27 Dec 24, 2022
A zero-dependency Python library for getting the Kubernetes token of a AWS EKS cluster

tokeks A zero-dependency Python library for getting the Kubernetes token of a AWS EKS cluster. No AWS CLI, third-party client or library (boto3, botoc

Chris Karageorgiou Kaneen 6 Nov 4, 2022
🔪 Block replies to viral tweets from users getting paid to promote useless products

This Tweet Took Off Ublock Origin filter list targeting long reply chains posted by twitter users who get paid to promote random products on viral twe

Xetera 12 Jan 14, 2022
Demonstrate how GitHub OIDC token getting should be included in boto3

boto3 should add direct support for AssumeRoleWithWebIdentity for GitHub Actions There is a aws-actions/configure-aws-credentials action that will get

Ben Kehoe 11 Aug 29, 2022