Skip to contents

Stream Process Large Files

Source: R/memory_management.R
stream_file.Rd

Stream Process Large Files

Usage

stream_file(
  file_path,
  process_fn,
  chunk_size = 10000,
  skip = 0,
  max_lines = NULL,
  progress = TRUE
)

Arguments

file_path

Path to the file to process

process_fn

Function to process each chunk/line

chunk_size

Number of lines to read at once

skip

Number of lines to skip at beginning of file

max_lines

Maximum number of lines to process (NULL = all)

progress

Logical, whether to show progress

Value

Result of processing