API reference

CSV

Fast, flexible reading and writing of delimited text.

Reading — CSV.File, CSV.read, CSV.lazy, CSV.Rows, CSV.Chunks. Writing — CSV.write, CSV.RowWriter. Diagnostics — CSV.problems, CSV.Problem, CSV.ParseError.

All public names live under the CSV namespace. Results do not depend on the number of threads.

CSV.File(source; keywords...) -> CSV.File

Read delimited data into an eager Tables.jl table. source can be a path or HTTP(S) URL, an IO, a Cmd, bytes, or a vector of sources. CSV.jl detects the delimiter and column types by default. Text uses DataStrings.DataString, pooling is off, and recoverable parse problems are available through CSV.problems. The default on_error=:warn prints one summary warning per read; on_error=:collect records problems silently; on_error=:error (or strict=true) throws a CSV.ParseError at the first problem.

Keywords

Header and rows:

• header=1: the header row number, false for Column1, Column2, ... names, a vector of names, or a vector of row numbers whose cells are joined.
• normalizenames=false: turn names into valid Julia identifiers.
• skipto: the first data row. footerskip=0: rows to omit at the end. limit: the maximum number of data rows.

Dialect:

• delim=nothing: a Char or String delimiter. nothing detects one of comma, tab, space, pipe, semicolon, or colon.
• quoted=true, quotechar='"', openquotechar, closequotechar, and escapechar (the closing quote by default).
• comment: skip rows that start with this string.
• ignoreemptyrows=true; ignorerepeated=false treats a run of delimiters as one.

Values:

• missingstring: a string or vector of strings that also read as missing. An unquoted empty field is always missing.
• truestrings, falsestrings: Bool spellings; a list replaces the defaults.
• dateformat: a format string or Dates.DateFormat, or a per-column dictionary of them.
• decimal='.', groupmark (digit grouping), and stripwhitespace=false.

Types and columns:

• types: one type, a vector with one type per column, or a dictionary keyed by column index, name, or Regex.
• typemap: a dictionary that replaces inferred types, such as Dict(Int64 => Float64).
• downcast=false: narrow Int64 columns to the smallest integer type that holds their values.
• select, drop: column indices, names, or a Bool mask; one name or index; a Regex; or an (i, name) -> Bool function.
• stringtype=DataStrings.DataString: the type of inferred text.
• pool=false: true, a ratio, (ratio, maximum_levels), or per-column policies.
• transpose=false: read input rows as output columns.
• scan: a Tables.Scan request for projection, filters, and row bounds.
• source: with a vector of sources, add a column naming each row's source.

Problems:

• on_error=:warn: :warn, :collect, or :error. strict=true means :error.
• maxproblems=10_000 (or maxwarnings): the number of problems retained.
• validate=true: require types, dateformat, and pool dictionary keys to match a column.

Performance:

• ntasks: the maximum number of parsing tasks. parallel=false uses one.
• chunkbytes: the target size of a structural-index chunk.
• nsample: rows sampled for type inference. samplebytes=65536: bytes sampled for delimiter detection.
• fastindex=true: the vector scanner; false uses the byte-at-a-time scanner.
• buffer_in_memory=false: copy a local file into memory instead of mapping it. prefetch=true: touch a mapped file's pages on worker threads.

See the Reading data guide for details.

CSV.read(source, sink; keywords...)

Parse with the same options as CSV.File, then call the Tables.jl sink, as in CSV.read("data.csv", DataFrame). The new columns are passed as Tables.CopiedColumns, so a sink that honors that marker can take ownership without another copy.

CSV.problems(file) -> Vector{CSV.Problem}

Return the parse problems a CSV.File retained, in source order: at most maxproblems of them (10,000 by default). Showing the file reports how many were recorded in total. Pass on_error=:collect to a reader to record problems without the summary warning, or on_error=:error to throw a CSV.ParseError instead.

CSV.Problem

One recoverable parse problem, as returned by CSV.problems. Fields: row (1-based data row; 0 for a file- or header-level problem), col (1-based column; 0 for a whole-row problem), pos (byte offset into the parsed bytes), kind, and message. The kinds are :short_row, :long_row, :invalid_value, :invalid_quoted_field, and :unclosed_quote.

CSV.ParseError <: Exception

Thrown by the readers under on_error=:error (or strict=true). problem is the source-earliest CSV.Problem in a File or Chunks batch, nproblems counts every problem found, and source labels the input. Rows reports the accessed cell's problem with nproblems=1.

CSV.lazy(source; keywords...) -> CSV.LazyFile

Build the quote-aware structural index and return a table whose cells parse when accessed. Column, cell, and Tables.jl column access are supported. CSV.File(lazyfile) performs a full typed parse without repeating the structural scan. This API retains the source bytes and index; it does not stream an unbounded input. List selection uses stable file order and removes duplicates. A later CSV.File(lazyfile) retains those visible columns and can only project them further. Ordinary text cells are zero-copy views. A long cell whose absolute source offset cannot fit the compact view word copies only that cell into a bounded backing buffer.

It takes the keywords of CSV.File for one source, except pool, typemap, downcast, transpose, scan, on_error, strict, maxproblems, ntasks, and nsample.

CSV.LazyFile

The indexed table that CSV.lazy returns. It holds the source bytes and the index, and no parsed values. names(lf), size(lf), and Tables.columnnames come from the index. lf.name and lf[:name] return a column view whose cells parse when read; lf.name[i] parses one cell. A cell is text (stringtype, DataString by default) or missing unless types requested a type for its column. A view keeps the source alive, so a LazyFile is a way to look before parsing, not a way to hold a large file cheaply. CSV.File(lf) runs the eager typed parse on the existing index and keeps the columns lf selected.

CSV.Rows(source; types=nothing, stringtype=DataStrings.DataString, keywords...)

Iterate Tables.jl row views; each cell parses when it is accessed. The source and an index of every row stay in memory, so length(rows) and names(rows) are known before iteration. reusebuffer is accepted and ignored. Invalid or malformed cells become missing by default; strict=true or on_error=:error throws a CSV.ParseError when the cell is accessed. Rows do not retain parse diagnostics, so use CSV.File when CSV.problems or a diagnostic cap is needed. select and drop (a list, one name, a Regex, or an (i, name) -> Bool function) project columns in stable file order. Other keywords are those of CSV.File for one source, except pool, typemap, downcast, transpose, scan, maxproblems, ntasks, and nsample.

CSV.Chunks(source; ntasks=Threads.nthreads(), chunkbytes=nothing, keywords...)

Iterate a source as CSV.File batches that all have the same column types, and serve them through Tables.partitions. Pooling is decided per batch; select and drop apply to every batch. With the default on_error=:warn, the first batch with parse problems prints one summary warning; every batch keeps its own CSV.problems.

One batch is one window of the source's data bytes. chunkbytes is the target size of that window and defaults to 64 MiB, or the whole source when it is smaller. A window ends at a row boundary, so a batch holds every row that starts within its bytes and one complete row can push it past chunkbytes. ntasks bounds the parallel work inside a window; it is not a batch count and does not affect the batch size.

Each window is indexed when its batch is produced and released with it, so a read holds one window's structural index rather than the whole source's. The source bytes stay in memory for later batches, and a returned batch owns its own. The constructor validates every row it will yield, before the first batch, to settle one schema. Value validation excludes the rows after limit: the window that holds the limit is indexed to its end, and the rows past the limit are dropped before any value is read. The pass starts over when a quote that did not start its field, or a timestamp that needs microseconds, invalidates what it settled. Its final run walks the windows iteration walks, so it also counts the batches and length is defined.

Other keywords are those of CSV.File for one source, except typemap, downcast, transpose, scan, maxwarnings, and nsample; pool takes one policy for every column.

CSV.write(sink, table; keywords...) -> sink

Write any Tables.jl table as delimited text to a path or IO and return sink. An IO is written at its current position. Column tables render in parallel; row tables and CSV.Chunks stream without being collected. The output is the same for every ntasks value. CSV.write(sink; keywords...) returns a function of the table, for table |> CSV.write(path).

Keywords

• delim=',': a character or a string such as "::".
• quotechar='"', openquotechar, closequotechar, and escapechar (the closing quote by default).
• quotestyle=:minimal: quote text only when it needs quotes; :all quotes all text, and :none throws for text that needs quotes. quotestrings=true means :all.
• newline='\n', missingstring="", and bom=false.
• writeheader=true (false when appending) and header, a vector of replacement names.
• append=false: append to a path instead of replacing it.
• compress=:auto: gzip a path ending in .gz; :gzip or :none force it on or off.
• floatformat: a Printf format such as "%.2f"; decimal='.'.
• dateformat: a Dates format for Date, DateTime, Timestamp, and Time values.
• transform: a (col, val) -> val function applied to each cell in row-major order.
• partition=false: write each of Tables.partitions(table) to its own sink, from a vector of sinks or a base path that gets _1, _2, ... appended (the generated paths are returned).
• ntasks=Threads.nthreads(): the maximum number of tasks.
• bufsize=4 MiB: the largest allowed row, in bytes.

See the Writing data guide for details.

CSV.RowWriter(table; keywords...)

Iterate the rows of table as CSV-formatted strings, the header first unless writeheader=false. It takes the dialect and formatting keywords of CSV.write; joining its rows gives the text CSV.write writes with the same keywords.