bigjsonvalue

package module
v0.0.0-...-57daaf7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 22, 2018 License: MIT Imports: 7 Imported by: 0

README

bigjsonvalue

Build status License GoDoc

bigjsonvalue replaces interface{} for decoding unknown JSON values

BigJSONValue and NatJSONValue are wrappers around the interface{} type, to force json.Unmarshal() to decode integer values as integers instead of float64. Main reason for this is float64 doesn't have enough precision to store exact values of large int64 or uint64 values. Instead of trying to unmarshal unknown JSON values into an interface{}, unmarshal into a BigJSONValue or NatJSONValue instead.

Example Usage

The impetus for bigjsonvalue is decoding the JSON encoded output from wal2json, which is an output plugin for PostgreSQL Logical-Decoding. This example uses the pgx library to connect to the wal2json output of PostgreSQL Logical-Decoding.

import (
        "context"
        "time"
        pgx "github.com/jackc/pgx"
        bjv "github.com/steampunkcoder/bigjsonvalue"
)

// WalOldKeys models the "oldkeys" map in a WAL change JSON
type WalOldKeys struct {
        KeyNames  []string           `json:"keynames"`
        KeyTypes  []string           `json:"keytypes"`
        KeyValues []bjv.BigJSONValue `json:"keyvalues"`
}

// WalChangeRec models a single change record in a WAL change JSON
type WalChangeRec struct {
        Kind         string             `json:"kind"`
        Schema       string             `json:"schema"`
        Table        string             `json:"table"`
        ColumnNames  []string           `json:"columnnames"`
        ColumnTypes  []string           `json:"columntypes"`
        ColumnValues []bjv.BigJSONValue `json:"columnvalues"`
        OldKeys      WalOldKeys         `json:"oldkeys"`
}

// WalChangeTx models an entire change list transaction in a WAL change JSON
// obtained from a pgx.WalMessage notification
type WalChangeTx struct {
        Changes []WalChangeRec `json:"change"`
}

func WalListenLoop() {
        slotName := "myslot"

        // Set walSenderTimeoutSecs to your PostgreSQL instance's wal_sender_timeout
        var walSenderTimeoutSecs uint64 = 60

        // Set startLsn to result of this PostgreSQL query:
        //   SELECT confirmed_flush_lsn FROM pg_replication_slots WHERE slot_name='myslot'
        // otherwise start from zero
        var startLsn uint64 = 0

        rConn, _ := pgx.ReplicationConnect(...)
        rConn.CreateReplicationSlot(slotName, "wal2json")
        rConn.StartReplication(slotName, startLsn, -1, ...)
        for {
                replyFlag = false
                timeoutCtx, ctxCancelFn := context.WithTimeout(context.Background(),
                        time.Second * walSenderTimeoutSecs / 2)
                defer ctxCancelFn()

                rMsg, err := rConn.WaitForReplicationMessage(timeoutCtx)
                ctxCancelFn()

                if err == context.Canceled {
                        break
                } else if err == context.DeadlineExceeded {
                        // PostgreSQL expects to be pinged by WAL client within wal_sender_timeout
                        // otherwise PostgreSQL will force close connection
                        replyFlag = true
                } else if rMsg.WalMessage != nil {
                        var chgTx WalChangeTx
                        json.Unmarshal(rMsg.WalMessage.WalData, &chgTx)
			if SuccessfullyProcessedWalChange(&chgTx) {
                                // Tell PostgreSQL we've successfully processed the
                                // LSN of this WAL change msg
                                replyFlag = true
                                startLsn = rMsg.WalMessage.WalStart
			}
                } else if rMsg.ServerHeartbeat != nil {
			if rMsg.ServerHeartbeat.ReplyRequested == 1 {
                                replyFlag = true
			}
                }

                if replyFlag {
                        sMsg, _ := pgx.NewStandbyStatus(startLsn)
                        rConn.SendStandbyStatus(sMsg)
                }
        }
}
API Docs

GoDoc

License

Released under MIT License License

Documentation

Overview

Package bigjsonvalue provides types to replace interface{} for decoding unknown JSON values

Index

Constants

View Source 
const (
	// JSONNumRegexpPat defines the regexp pattern for matching JSON numbers
	// (integers or floats) based on http://json.org
	JSONNumRegexpPat = `^-?\d+(\.\d+)?([eE][-+]?\d+)?$`
)

Package constants

Variables

View Source 
var (
	// ErrInvalidJSON defines the invalid JSON error
	ErrInvalidJSON = errors.New("invalid JSON")

	// ErrNotImplemented defines the not-implemented error
	ErrNotImplemented = errors.New("not implemented")
)

Package errors

Functions

This section is empty.

Types

type BigJSONValue 

type BigJSONValue struct {
	// contains filtered or unexported fields
}

BigJSONValue is wrapper around interface{} type to force json.Unmarshal() to decode integer values as big.Int instead of float64. The problem with float64 is that it doesn't have enough precision to store exact values of large int64 and uint64 values. Instead of trying to unmarshal JSON into an interface{}, unmarshal into a BigJSONValue instead.

Compared to NatJSONValue, BigJSONValue uses big.Int and big.Float to store arbitrary-precision numbers, but is slower than NatJSONValue.

func (*BigJSONValue) BigFloat 

func (bjv *BigJSONValue) BigFloat() big.Float

BigFloat returns the underlying big.Float value. Panics with runtime error if not a big.Float.

func (*BigJSONValue) BigInt 

func (bjv *BigJSONValue) BigInt() big.Int

BigInt returns the underlying big.Int value. Panics with runtime error if not a big.Int.

func (*BigJSONValue) Bool 

func (bjv *BigJSONValue) Bool() bool

Bool returns the underlying bool value. Panics with runtime error if not a bool.

func (*BigJSONValue) DecodeJSONValue 

func (bjv *BigJSONValue) DecodeJSONValue(text string) (*BigJSONValue, error)

DecodeJSONValue decodes a JSON value, and returns itself. Results are undefined if error is returned.

The text "null" is decoded as a nil value.

The text "true" and "false" are decoded as bool values.

Text surrounded by double-quotes are decoded as string values.

Number text containing period "." or the letters "e" or "E" are decoded as big.Float values.

Otherwise, number text is decoded as big.Int values. Whether text is considered a number is based on http://json.org

func (*BigJSONValue) IsBigFloat 

func (bjv *BigJSONValue) IsBigFloat() bool

IsBigFloat returns true if value is a big.Float.

func (*BigJSONValue) IsBigInt 

func (bjv *BigJSONValue) IsBigInt() bool

IsBigInt returns true if value is a big.Int.

func (*BigJSONValue) IsBool 

func (bjv *BigJSONValue) IsBool() bool

IsBool returns true if value is a bool.

func (*BigJSONValue) IsNil 

func (bjv *BigJSONValue) IsNil() bool

IsNil returns true if value is nil.

func (*BigJSONValue) IsString 

func (bjv *BigJSONValue) IsString() bool

IsString returns true if value is a string.

func (*BigJSONValue) Kind 

func (bjv *BigJSONValue) Kind() Kind

Kind returns the kind of BigJSONValue it is holding:

Returns Bool if value is a bool.

Returns String if value is a string.

Returns BigInt if value is a big.Int.

Returns BigFloat if value is a big.Float.

Otherwise returns Nil.

func (*BigJSONValue) String 

func (bjv *BigJSONValue) String() string

String implements fmt.Stringer interface for BigJSONValue.

Bool values return "true" or "false".

String values return as-is (no surround double-quotes are added).

Number values return with as much precision as possible.

Nil values return "nil".

func (*BigJSONValue) UnmarshalJSON 

func (bjv *BigJSONValue) UnmarshalJSON(text []byte) error

UnmarshalJSON implements the json.Unmarshaler interface for BigJSONValue

func (*BigJSONValue) Value 

func (bjv *BigJSONValue) Value() interface{}

Value returns the underlying interface{} value that is being wrapped.

type Kind 

type Kind uint

Kind enumerates the kind of underlying value being held by the wrapped interface{} value 

const (
	Nil Kind = iota
	Bool
	String
	Int64
	Uint64
	Float64
	BigInt
	BigFloat
)

Kind enumeration constants

func (Kind) String 

func (k Kind) String() string

String implements fmt.Stringer interface for Kind

type NatJSONValue 

type NatJSONValue struct {
	// contains filtered or unexported fields
}

NatJSONValue is wrapper around interface{} type to force json.Unmarshal() to decode integer values as int64 or uint64 instead of float64. The problem with float64 is that it doesn't have enough precision to store exact values of large int64 and uint64 values. Instead of trying to unmarshal JSON into an interface{}, unmarshal into a NatJSONValue instead.

Compared to BigJSONValue, NatJSONValue uses native Golang number types int64, uint64, and float64 to store numbers, so is faster than BigJSONValue.

func (*NatJSONValue) Bool 

func (njv *NatJSONValue) Bool() bool

Bool returns the underlying bool value. Panics with runtime error if not a bool.

func (*NatJSONValue) DecodeJSONValue 

func (njv *NatJSONValue) DecodeJSONValue(text string) (*NatJSONValue, error)

DecodeJSONValue decodes a JSON value, and returns itself. Results are undefined if error is returned.

The text "null" is decoded as a nil value.

The text "true" and "false" are decoded as bool values.

Text surrounded by double-quotes are decoded as string values.

Number text containing period "." or the letters "e" or "E" are decoded as float64 values.

Otherwise, number text is decoded as int64 for negative values or uint64 for positive values. Whether text is considered a number is based on http://json.org

func (*NatJSONValue) Float64 

func (njv *NatJSONValue) Float64() float64

Float64 returns the underlying float64 value. Panics with runtime error if not a float64.

func (*NatJSONValue) Int64 

func (njv *NatJSONValue) Int64() int64

Int64 returns the underlying int64 value. Panics with runtime error if not a int64.

func (*NatJSONValue) IsBool 

func (njv *NatJSONValue) IsBool() bool

IsBool returns true if value is a bool.

func (*NatJSONValue) IsFloat64 

func (njv *NatJSONValue) IsFloat64() bool

IsFloat64 returns true if value is a float64.

func (*NatJSONValue) IsInt64 

func (njv *NatJSONValue) IsInt64() bool

IsInt64 returns true if value is a int64.

func (*NatJSONValue) IsNil 

func (njv *NatJSONValue) IsNil() bool

IsNil returns true if value is nil.

func (*NatJSONValue) IsString 

func (njv *NatJSONValue) IsString() bool

IsString returns true if value is a string.

func (*NatJSONValue) IsUint64 

func (njv *NatJSONValue) IsUint64() bool

IsUint64 returns true if value is a uint64.

func (*NatJSONValue) Kind 

func (njv *NatJSONValue) Kind() Kind

Kind returns the kind of NatJSONValue it is holding:

Returns Bool if value is a bool.

Returns String if value is a string.

Returns Int64 if value is a int64.

Returns Uint64 if value is a uint64.

Returns Float64 if value is a float64.

Otherwise returns Nil.

func (*NatJSONValue) String 

func (njv *NatJSONValue) String() string

String implements fmt.Stringer interface for NatJSONValue.

Bool values return "true" or "false".

String values return as-is (no surround double-quotes are added).

Number values return with as much precision as possible.

Nil values return "nil".

func (*NatJSONValue) Uint64 

func (njv *NatJSONValue) Uint64() uint64

Uint64 returns the underlying uint64 value. Panics with runtime error if not a uint64.

func (*NatJSONValue) UnmarshalJSON 

func (njv *NatJSONValue) UnmarshalJSON(text []byte) error

UnmarshalJSON implements the json.Unmarshaler interface for NatJSONValue

func (*NatJSONValue) Value 

func (njv *NatJSONValue) Value() interface{}

Value returns the underlying interface{} value that is being wrapped.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.