Read large text file

Read large text file by splitting columns.

big_fread2(file, nb_parts = NULL, .transform = identity,
  .combine = cbind_df, skip = 0, select = NULL, progress = FALSE,
  part_size = 500 * 1024^2, ...)

Arguments

file

Path to file that you want to read.

nb_parts

Number of parts in which to split reading (and transforming). Parts are referring to blocks of selected columns. Default uses part_size to set a good value.

.transform

Function to transform each data frame corresponding to each block of selected columns. Default doesn't change anything.

.combine

Function to combine results (list of data frames).

skip

Number of lines to skip at the beginning of file.

select

Indices of columns to keep (sorted). Default keeps them all.

progress

Show progress? Default is FALSE.

part_size

Size of the parts if nb_parts is not supplied. Default is 500 * 1024^2 (500 MB).

...

Other arguments to be passed to data.table::fread, excepted input, file, skip, select and showProgress.

Value

The outputs of fread2 + .transform, combined with .combine.