This tutorial explains how to scrape Twitter lists in Windows and export list members for free. Here I will talk about 3 different tools to bulk download members from multiple Twitter lists. The tools that I have listed here take a list of a user or the URL of the list to fetch the list members. After fetching the list of members, these tools save the list as JSON or Excel file on your PC. In the list, you can see the Twitter handle and profile URL of the list members. And if you want to get the CSV out of that JSON files, then you can try some JSON to CSV converters software or websites to do that.
If you generally use Twitter lists, then you may want to get the members in the lists on your PC. That can be useful in many places, say if you are in a digital marketing organization, then surely you will need to do that. And in the following list, I will talk about some free tools that you can easily run on your Windows PC. Some of them are command line tools, while some of them come with a nice user interface.
How to Scrape Twitter Lists in Windows, Export List Members?
To demonstrate how to Scrape Twitter Lists in Windows, I will list three different tools in the different sections. And based on your interest, you can try any one from this list and export the Twitter list members to a file.
Twitter List Scrapper
Twitter List Scrapper is a free software for Windows to scrape Twitter lists in bulk. It can download the profile URL and their Twitter handle as a CSV file and save it on your PC. You can specify multiple URLs of the different lists in this software and it will process them all. It will go through each URL you specify and downloads the name of the members and save to a CSV file.
The software is very simple and doesn’t have other features in it. It is a dedicated software to scrap Twitter lists right from the desktop. And it is very simple to get started with this. Just run the software and then enter the URLs of the Twitter lists you want to scrape in the box and then hit the Scrape button. After that, it will start processing the lists and will ask you where you want to save the CSV file containing usernames of the people from those lists.
Scrape Twitter Lists using a Python Tool: Listslurper
Apart from using a GUI based Windows software, you can use a small and simple Python tool to do the same. The tool is called Listslurper and it uses a configuration file to fetch a list of members in different Twitter lists. You can use this tool to fetch Twitter handles and their id from multiple lists concurrently and it places them in JSON format. All you have to do is just run the tools once and it will automatically take care of the rest of the process. If you have some experience of Python, then you will find it very simple to use this tool. And if you don’t know anything about Python, then still, you will understand how to use this.
Before starting with this tool, make sure that you have Python(2.7) installed on your PC. Also, you will need Twitter API credentials like API key, API Secret, Access Token and Access Token Secret. If you have a working Twitter account, then you can easily create an app by seeing the steps mentioned here.
Here are the steps to scrape Twitter lists using Listslurper.
Step 1: Grab the ZIP file of Listslurper from here. After that, extract the contents and then open the “listslurper.py” file. And then enter the Twitter API credentials that you have already generated (line 6 to 9).
Step 2: In the contents of the Listslurper, you will find a “lists.json” file. In this file, you have to add the URLs of the lists whose member you want to download. Simply place the list name and URLs as shown in the below picture.
Step 3: Now, open command prompt and simply run the “Listslurper.py” file. When it has done, you will see a JSON file will be created in with the name of the target Twitter list. You can open that in any text editor. And you can run the file as shown below.
python slurper.py or py -2 slurper.py
In this way, you can use this simple tool to scrape Twitter lists in bulk. After getting the Twitter handles from the lists, you can do whatever you want.
Scrape Twitter
Scrape Twitter is another free tool that you can use to easily scrap a single Twitter list. However, this tool works in a different way. Instead of only list members, it can scrap the whole list’s timeline in JSON format. And in the scraped data, you can see a lot of parameters like Twitter handle of the user, tweet text, time of the tweet, hashtags, retweet status, pinned status, and a lot others. You can see all this data and then you can do whatever you want.
To run this tool, you only need Node.js installed on your PC. And the best part about this tool is that, you don’t need the Twitter API credentials here. It uses some other approach to scrape the given Twitter list and save the result.
Here are the steps to scrape Twitter using Scrape Twitter tool for free.
Step 1: Open a command prompt with admin rights at any location on your PC and run this command. This will install the tool globally on your PC and you can access it from anywhere with a simple command.
npm install -g scrape-twitter
Step 2: After the above command finishes, you can start using the tool. To start scraping the Twitter lists, you can run this command and in the specified format.
scrape-twitter list "Twitter handle of the user" "Name of the list" > test.json
After the successful completion of the command, you will see a “test.json” file will be created in the current working directory (in which the current CMD is open). And you can open that and you will see the JSON response in that.
This way, you can use the Scrape Twitter tool to Scrape Twitter lists. By default, it can process one list at a time, but you can use to scrape multiple Twitter lists. For that, you can make batch file by entering the above command.
Final Thoughts
These are the best free tools that you can use to scrape Twitter lists in bulk. All the tools above worked fine for me and were able to scrape various Twitter lists in a few seconds. So, if you are looking for some nice tools to scrape Twitter lists, then this long tutorial will definitely come in handy. If you want simplicity, then you can go with the Twitter List Scrapper. And if you want some powerful tools, then use the two command line tools that I have listed above.
