Preprocess review text for topic modeling — preprocess

This function preprocesses the review text by optionally filtering non-English reviews, removing punctuation, converting to lowercase, removing stopwords, and stemming.

Usage

preprocess_reviews(reviews, english_only = TRUE)

Arguments

reviews: A data frame containing the scraped reviews
english_only: A logical value indicating whether to filter out non-English reviews. Default is TRUE

Value

A list containing the following elements:

corpus: The preprocessed corpus object.
dtm: The document-term matrix.
filtered_reviews: The filtered reviews data frame.

Examples

# \donttest{
# Create a temporary file with sample book IDs
temp_file <- tempfile(fileext = ".txt")
writeLines(c("1420", "2767052", "10210"), temp_file)

# Scrape reviews
reviews <- scrape_reviews(temp_file, num_reviews = 5, use_parallel = FALSE)
#> Total book IDs to process: 3
#> 2024-09-03 16:17:00.707278 scrape_goodreads_reviews: Completed! All book reviews extracted
#> Scraping run time = 8.23771452903748
#> Total books processed: 3

# Preprocess the reviews
preprocessed <- preprocess_reviews(reviews, english_only = TRUE)

# Print the document-term matrix
print(preprocessed$dtm)
#> <<DocumentTermMatrix (documents: 13, terms: 1799)>>
#> Non-/sparse entries: 2929/20458
#> Sparsity           : 87%
#> Maximal term length: 73
#> Weighting          : term frequency (tf)

# Clean up: remove the temporary file
file.remove(temp_file)
#> [1] TRUE
# }