AlgoMaster Logo

CSV Encoding

Low Priority18 min readUpdated June 6, 2026

CSV is the lingua franca of bulk data handoffs: analysts pull product exports into Excel, finance teams hand off daily order reports as .csv files, and partners ship inventory updates as comma-separated rows. Go's standard library ships encoding/csv, a small, no-frills package that reads and writes CSV without trying to be clever about it. This lesson covers the Reader and Writer types, their configuration knobs, how to map between records and structs by hand, and the streaming patterns that make large files tractable.

Why encoding/csv Looks Different from encoding/json

The encoding/json package maps directly between JSON and Go structs. You hand it a struct, it walks the fields by name (with help from tags), and it produces or consumes JSON in one call. CSV doesn't work that way in Go's standard library, and the reason is the format itself.

A CSV file is a stream of records, where each record is a slice of strings. The format has no type system, no nested objects, no canonical way to name fields, and no agreed-upon mechanism for the header row. The optional header line, if present, is just another record of strings sitting at the top of the file. Different tools quote differently, escape differently, and even disagree on the line ending.

So encoding/csv exposes a record-oriented API. The reader gives you []string. The writer accepts []string. Mapping those slices to and from your Product or Order structs is your job, usually through a small pair of helper functions. This sounds like extra work, but the upside is full control over column order, type conversions, and error handling for the inevitable malformed row.

The reader handles the file-to-records part. You handle the records-to-structs part. Writing flows in the reverse direction with the same separation of concerns.

Writing CSV

A writer is created with csv.NewWriter(io.Writer). The returned *csv.Writer is buffered, so the bytes don't reach the underlying writer until you call Flush(). The most common version of "my CSV file is empty" is a missing flush.

A Minimal Export

Suppose the store wants to send the analyst a list of products with their prices and stock. The columns are id, name, price, stock. Here's the smallest program that writes such a file to standard output:

WriteAll does three things in one call: it writes every record, flushes the buffer, and returns the first error it encountered. If you use it, you don't have to call Flush() separately. Most batch-style exports use WriteAll for exactly this reason.

The header row is just another record. The package doesn't know or care that the first row is a header. If you want to include one, write it. If you don't, don't.

Writing One Record at a Time

For a large export where you can't hold all the rows in memory at once, write records one at a time and flush at the end.

Write buffers the record but doesn't return errors from the underlying writer immediately, because of the buffering. After the final Flush(), you call w.Error() to see whether any deferred write failed. This is the canonical pattern for csv.Writer: write everything, flush, then check the error.

The productToRecord helper is doing the manual struct-to-record mapping. Note that everything ends up as a string, including the price and the stock count. CSV has no native numeric type, so the writer converts with fmt.Sprintf. The %.2f format keeps the price at two decimal places, which is what an analyst opening the file in Excel would expect.

Special Characters Get Quoted Automatically

A product name with a comma in it, a description containing a literal double-quote, a multi-line note: the writer handles all three without any extra code.

Three things happened. The comma inside Coffee, Ground made the writer wrap the field in double quotes. The literal " in the monitor name got doubled to "" inside a quoted field, which is how CSV escapes quotes. The newline inside the gift card note created a multi-line record, with the entire field wrapped in quotes so a reader knows the line break belongs to the value, not the row.

You don't have to do anything to get this behavior. The writer applies RFC 4180 quoting whenever a field contains a delimiter, a double quote, or a newline. Fields without those characters are written bare.

Writer Configuration

Two fields on *csv.Writer change the output format:

FieldDefaultEffect
Comma','Field delimiter. Set to '\t' to write TSV, ';' to match European Excel locales.
UseCRLFfalseWhen true, lines end with \r\n instead of \n. Use this for Windows-only consumers or strict RFC 4180 output.

Set these fields right after NewWriter and before any Write call. Changing them mid-stream produces a file with mixed conventions, which most consumers can't parse.

Reading CSV

The reader API mirrors the writer. csv.NewReader(io.Reader) returns a *csv.Reader that produces records, either one at a time with Read() or all at once with ReadAll().

Reading the Whole File

The simplest case: a small file fits in memory, so read it all and loop over the records.

ReadAll returns [][]string. The first sub-slice is the header row if your file has one (the reader doesn't know that; it's just the first record). The rest are data rows. Index into each record by column position.

Streaming Read

For files of unknown size, or any time you don't need every row in memory at once, loop over Read() until it returns io.EOF. This is the pattern Go uses for almost all stream readers.

The loop terminates when Read() returns io.EOF. Use errors.Is(err, io.EOF) rather than err == io.EOF: the comparison works today, but errors.Is is the idiomatic form and survives wrapping. Any other non-nil error signals a malformed row, which you handle separately.

The header is read and thrown away on the first iteration. The reader doesn't have a built-in concept of a header row, so you handle it by reading once and ignoring the result. If you wanted to use the header to build a column-name-to-index map, you'd capture that first record instead of discarding it.

Records to Structs

The reader gives you []string. Almost every real program wants a struct. Here's the mirror of the productToRecord helper from earlier:

The helper validates the field count first, because a short record would cause the parser to index out of bounds. The price and stock conversions use strconv.ParseFloat and strconv.Atoi, the standard library tools for string-to-number parsing. Both return errors that get wrapped with %w so the caller can use errors.Is and errors.As on them.

The loop continues on parse errors rather than aborting. Real-world CSVs almost always have a few junk rows. A reasonable default for an import job is to log the bad rows, keep the good ones, and report the count at the end. If your use case demands all-or-nothing imports, return early on the first error instead.

The reader is one half of the pipeline. The other half is your mapping helper, which knows the column order, validates types, and decides what to do with bad records. Splitting these responsibilities makes the code easier to test: you can feed recordToProduct arbitrary []string values in a unit test without touching a file.

Reader Configuration

Most of the reader's behavior can be tuned through fields on *csv.Reader. Set them before the first call to Read or ReadAll. Mid-stream changes are undefined.

FieldDefaultPurpose
Comma','Field delimiter. Set to '\t' for TSV, ';' for some European Excel exports.
Comment0 (off)Lines that start with this rune are skipped entirely. Useful for vendor files that include #-prefixed metadata.
FieldsPerRecord0Field count enforcement. 0 (default) means "match the first record's field count". -1 means "allow any count". A positive number enforces exactly that many fields per row.
LazyQuotesfalseWhen true, the reader accepts files where bare quotes appear in unquoted fields. Use only when you have no control over the producer.
TrimLeadingSpacefalseWhen true, leading whitespace is stripped from each field. Some legacy exports add a space after every comma.
ReuseRecordfalsePerformance flag. When true, Read returns the same backing slice each call, avoiding per-row allocation. The contents are valid only until the next Read.

TSV with a Comment Line

A common vendor format is tab-separated values with comment lines for metadata.

The # lines are skipped because of r.Comment = '#'. The tab delimiter is set by r.Comma. Notice that the comment behavior applies only to whole lines whose first character is the comment rune. A # in the middle of a field is not special.

FieldsPerRecord Pitfalls

The default value of FieldsPerRecord is 0, which sounds like "no enforcement" but actually means "match the first record's count." This catches a class of import bugs where someone adds a column to the header and forgets to add it to the data rows.

If you genuinely want to allow ragged rows (some vendors do this with trailing-optional columns), set r.FieldsPerRecord = -1 and add your own per-row length checks downstream.

The trade-off is real. Strict mode catches mistakes; lax mode tolerates messy data. Pick the one that matches the producer you're consuming from.

Performance: ReuseRecord

The default behavior of Read allocates a fresh []string for every record. For a one-megabyte file that's negligible. For a one-gigabyte file processed at line-rate, those allocations are a measurable cost.

ReuseRecord = true tells the reader to recycle the same backing slice across calls. Subsequent calls overwrite the previous record in place. This eliminates the per-record allocation but adds a sharp constraint: you must process each record (or copy it) before calling Read again.

If you need to keep records around (for example, to sort them after the file is closed), copy each one with append([]string(nil), rec...) before storing it. Otherwise, every retained "record" ends up pointing at the same shared buffer and will contain whatever the last Read wrote.

Working with Files

The examples so far used strings.NewReader and os.Stdout to keep them runnable. Real exports and imports use os.File. The wiring is the same; the only differences are remembering to close the file and handling I/O errors that happen during the write.

Bulk Product Import from a CSV File

A daily job ingests a partner's product feed and inserts the products into the catalog. The file is on disk, the count is unknown, and bad rows should be logged but not abort the import.

The shape is unchanged from the in-memory version. The reader doesn't care whether its io.Reader is a *os.File, a *strings.Reader, or a network connection. This is the payoff of programming against io.Reader instead of file-specific APIs.

Exporting Orders to a CSV File

The reverse direction: a daily report job dumps yesterday's orders for the finance team.

The Flush plus Error pair is the canonical close-out for a CSV writer. Errors deferred by the buffering only surface during or after the flush, so checking w.Error() separately catches I/O problems that w.Write couldn't report.

Two small details about the format. The timestamp is written as RFC 3339 because that's a string format every consumer can parse unambiguously. The total uses strconv.FormatFloat with two decimal places, which is exactly the same output as fmt.Sprintf("%.2f", ...) but slightly faster in tight loops because it avoids the format-string parser.

Customer Export with Multi-Line Addresses

CSV doesn't require single-line records. A multi-line address survives the round trip as long as the writer quotes it and the reader uses default quoting rules.

Both addresses end up wrapped in double quotes because of the embedded newlines, and the second one also contains a comma that would otherwise be a delimiter. The writer applies both protections automatically.

Edge Cases Worth Knowing

A few format details come up regularly.

  • Embedded commas: wrapped in double quotes automatically by the writer and parsed back transparently by the reader.
  • Embedded quotes: a literal " becomes "" inside a quoted field. The round trip "He said ""hi""" becomes the in-memory string He said "hi".
  • Embedded newlines: newlines inside a quoted field belong to the value. The record spans multiple physical lines but counts as one logical record.
  • Empty fields: two delimiters in a row produce one empty-string field. P-1001,,29.99 parses to three fields, not two. There is no way to distinguish "empty" from "missing" in the format.
  • Type system: there is none. Every value is a string on the wire. Convert with strconv or time.Parse on the way in, strconv.Format* or time.Format on the way out.
  • Headers are just rows: the reader doesn't know the first line is special. Read it and discard, or capture it and build a map[string]int if you want column-name access.
  • Line endings: both \n and \r\n are accepted by the reader. The writer emits \n by default; set UseCRLF = true for Windows-style endings.

Round-Tripping Through a Buffer

A useful trick for testing CSV code is to write into a bytes.Buffer and read straight back out of it. The buffer behaves as both io.Writer and io.Reader, so the writer and reader from earlier sections compose without touching the disk.

The same structure works for unit tests around any productToRecord/recordToProduct pair: build the expected CSV in memory, feed it through the round trip, and assert on the result. No temporary files, no cleanup.

Quiz

CSV Encoding Quiz

10 quizzes