I'm hoping someone here with more experience can point me in the right direction because I've been going in circles trying to figure this out. I'm working on a personal music discovery project. I want to search approximately five years of discussions in a particular subreddit and build a massive list of artists and bands that people have recommended over the years. The biggest challenge is that I don't want to rely on Reddit's limited search results or just collect the most popular posts. I'm trying to dig as deep as possible, including older discussions, overlooked threads, obscure recommendations, and names buried in comment replies. Ideally, I'd like to: - Search hundreds or thousands of historical threads using multiple keywords. - Retrieve complete comment sections, including nested replies. - Identify recommendation-related discussions while filtering out unrelated posts. - Export the collected information into CSV files for further processing. - Extract and deduplicate artist names to create one comprehensive list. Here's what I've already tried: I installed Python, PRAW, and Selenium on my Windows computer. Unfortunately, Reddit requires API approval for my intended use, and my application was denied. I also tried Selenium with Firefox, but Reddit returned a network security block. I've looked into a few open-source projects, but I'm unsure which ones are reliable or currently working. What I'm looking for is a completely free solution that can handle a large amount of historical data. I don't mind letting a script run for several hours if necessary. Has anyone successfully collected several years of Reddit posts and comments recently? Are there any open-source tools, public Reddit archives, downloadable datasets, or other legitimate approaches you'd recommend? I'm particularly interested in methods that can retrieve older discussions and their full comment trees rather than stopping after the first few hundred results. This is strictly for personal music discovery, not commercial use or collecting personal information. I'd really appreciate hearing what has actually worked for people rather than just theoretical suggestions. Thanks in advance!   submitted by   /u/xmeandix [link]   [comments]