Skip to content

josephcopenhaver/csv-go

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

csv-go

csv-go

Go Report Card tests code-coverage License: MIT

This package is a highly flexible and performant single threaded csv stream reader and writer. It opts for strictness with nearly all options off by default. Using the option functions pattern on Reader and Writer creation ensures extreme flexibility can be offered while configuration can be validated up-front in cold paths. This creates an immutable, clear execution of the csv file/stream parsing strategy. It has been battle tested thoroughly in production contexts for both correctness and speed so feel free to use in any way you like.

The reader is also more performant than the standard go csv package when compared in an apples-to-apples configuration between the two. I expect mileage here to vary over time. My primary goal with this lib was to solve my own edge case problems like suspect-encodings/loose-rules and offer something back more aligned with others that think like myself with regard to reducing allocations, GC pause, and increasing efficiency.

package main

// this is a toy example that reads a csv file and writes to another

import (
	"os"

	"github.com/josephcopenhaver/csv-go/v2"
)

func main() {
	r, err := os.Open("input.csv")
	if err != nil {
		panic(err)
	}
	defer r.Close()

	cr, err := csv.NewReader(
		csv.ReaderOpts().Reader(r),
		// by default quotes have no meaning
		// so must be specified to match RFC 4180
		// csv.ReaderOpts().Quote('"'),
	)
	if err != nil {
		panic(err)
	}
	defer cr.Close()

	w, err := os.Create("output.csv")
	if err != nil {
		panic(err)
	}
	defer w.Close()

	cw, err := csv.NewWriter(
		csv.WriterOpts().Writer(w),
	)
	if err != nil {
		panic(err)
	}
	defer cw.Close()

	for row := range cr.IntoIter() {
		if _, err := cw.WriteRow(row...); err != nil {
			panic(err)
		}
	}
	if err := cr.Err(); err != nil {
		panic(err)
	}
}

See the Reader and Writer examples for more in-depth usages.

Reader Features

Name option(s)
Zero allocations during processing BorrowRow + BorrowFields + InitialRecordBuffer + InitialRecordBufferSize + NumFields
Format Specification Comment + CommentsAllowedAfterStartOfRecords + Escape + FieldSeparator + Quote + RecordSeparator + NumFields
Format Discovery DiscoverRecordSeparator
Data Loss Prevention ClearFreedDataMemory
Byte Order Marker Support RemoveByteOrderMarker + ErrorOnNoByteOrderMarker
Headers Support ExpectHeaders + RemoveHeaderRow + TrimHeaders
Reader Buffer tuning ReaderBuffer + ReaderBufferSize
Format Validation ErrorOnNoRows + ErrorOnNewlineInUnquotedField + ErrorOnQuotesInUnquotedField
Security Limits planned

Writer Features

Name option(s)
Zero allocations planned
Header and Comment Specification CommentRune + CommentLines + IncludeByteOrderMarker + Headers + TrimHeaders
Format Specification Escape + FieldSeparator + Quote + RecordSeparator + NumFields
Data Loss Prevention ClearFreedDataMemory
Encoding Validation ErrorOnNonUTF8
Security Limits planned

CHANGELOG


Go Reference

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

No packages published