csv-go
This package is a highly flexible and performant single threaded csv stream reader and writer. It opts for strictness with nearly all options off by default. Using the option functions pattern on Reader and Writer creation ensures extreme flexibility can be offered while configuration can be validated up-front in cold paths. This creates an immutable, clear execution of the csv file/stream parsing strategy. It has been battle tested thoroughly in production contexts for both correctness and speed so feel free to use in any way you like.
The reader is also more performant than the standard go csv package when compared in an apples-to-apples configuration between the two. I expect mileage here to vary over time. My primary goal with this lib was to solve my own edge case problems like suspect-encodings/loose-rules and offer something back more aligned with others that think like myself with regard to reducing allocations, GC pause, and increasing efficiency.
package main
// this is a toy example that reads a csv file and writes to another
import (
"os"
"github.com/josephcopenhaver/csv-go/v2"
)
func main() {
r, err := os.Open("input.csv")
if err != nil {
panic(err)
}
defer r.Close()
cr, err := csv.NewReader(
csv.ReaderOpts().Reader(r),
// by default quotes have no meaning
// so must be specified to match RFC 4180
// csv.ReaderOpts().Quote('"'),
)
if err != nil {
panic(err)
}
defer cr.Close()
w, err := os.Create("output.csv")
if err != nil {
panic(err)
}
defer w.Close()
cw, err := csv.NewWriter(
csv.WriterOpts().Writer(w),
)
if err != nil {
panic(err)
}
defer cw.Close()
for row := range cr.IntoIter() {
if _, err := cw.WriteRow(row...); err != nil {
panic(err)
}
}
if err := cr.Err(); err != nil {
panic(err)
}
}
See the Reader and Writer examples for more in-depth usages.
Name | option(s) |
---|---|
Zero allocations during processing | BorrowRow + BorrowFields + InitialRecordBuffer + InitialRecordBufferSize + NumFields |
Format Specification | Comment + CommentsAllowedAfterStartOfRecords + Escape + FieldSeparator + Quote + RecordSeparator + NumFields |
Format Discovery | DiscoverRecordSeparator |
Data Loss Prevention | ClearFreedDataMemory |
Byte Order Marker Support | RemoveByteOrderMarker + ErrorOnNoByteOrderMarker |
Headers Support | ExpectHeaders + RemoveHeaderRow + TrimHeaders |
Reader Buffer tuning | ReaderBuffer + ReaderBufferSize |
Format Validation | ErrorOnNoRows + ErrorOnNewlineInUnquotedField + ErrorOnQuotesInUnquotedField |
Security Limits | planned |
Name | option(s) |
---|---|
Zero allocations | planned |
Header and Comment Specification | CommentRune + CommentLines + IncludeByteOrderMarker + Headers + TrimHeaders |
Format Specification | Escape + FieldSeparator + Quote + RecordSeparator + NumFields |
Data Loss Prevention | ClearFreedDataMemory |
Encoding Validation | ErrorOnNonUTF8 |
Security Limits | planned |