Skip to content

Initial Dependabot Job JSON Schema #416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

JamieMagee
Copy link
Contributor

The Dependabot job format is shared between multiple different projects under the Dependabot umbrella. There are at least 3:

However, there is no common source of truth for all of them, which can lead to inconsistencies and bugs, like #408 to name a recent example.

I'm proposing using a JSON schema as the source of truth for the format, and generating the language specific serializers and deserializers from the JSON schema. That way we can ensure that all different implementations share the same format, and that the format is well documented.

This PR adds the initial JSON schema for the format.

@JamieMagee JamieMagee requested a review from a team as a code owner March 26, 2025 04:25
@jakecoffman
Copy link
Member

Yeah good idea! Modeling the job in the CLI was the first step so it makes sense to formalize it even more.

In this PR can you generate a new model package based off of this schema? That will give us confidence seeing the smoke tests still passing. Then we will know the model is correct or close enough!

@JamieMagee
Copy link
Contributor Author

Using omissis/go-jsonschema, which seems to be the most popular JSON Schema to Go tool, I get

Details
// Code generated by github.com/atombender/go-jsonschema, DO NOT EDIT.

package model

import "time"

type Advisory struct {
	// AffectedVersions corresponds to the JSON schema field "affected-versions".
	AffectedVersions []string `json:"affected-versions" yaml:"affected-versions" mapstructure:"affected-versions"`

	// DependencyName corresponds to the JSON schema field "dependency-name".
	DependencyName string `json:"dependency-name" yaml:"dependency-name" mapstructure:"dependency-name"`

	// PatchedVersions corresponds to the JSON schema field "patched-versions".
	PatchedVersions []string `json:"patched-versions" yaml:"patched-versions" mapstructure:"patched-versions"`

	// UnaffectedVersions corresponds to the JSON schema field "unaffected-versions".
	UnaffectedVersions []string `json:"unaffected-versions" yaml:"unaffected-versions" mapstructure:"unaffected-versions"`
}

type Allowed struct {
	// DependencyName corresponds to the JSON schema field "dependency-name".
	DependencyName *string `json:"dependency-name,omitempty" yaml:"dependency-name,omitempty" mapstructure:"dependency-name,omitempty"`

	// DependencyType corresponds to the JSON schema field "dependency-type".
	DependencyType *string `json:"dependency-type,omitempty" yaml:"dependency-type,omitempty" mapstructure:"dependency-type,omitempty"`

	// UpdateType corresponds to the JSON schema field "update-type".
	UpdateType *string `json:"update-type,omitempty" yaml:"update-type,omitempty" mapstructure:"update-type,omitempty"`
}

type CommitOptions struct {
	// IncludeScope corresponds to the JSON schema field "include-scope".
	IncludeScope *bool `json:"include-scope,omitempty" yaml:"include-scope,omitempty" mapstructure:"include-scope,omitempty"`

	// Prefix corresponds to the JSON schema field "prefix".
	Prefix *string `json:"prefix,omitempty" yaml:"prefix,omitempty" mapstructure:"prefix,omitempty"`

	// PrefixDevelopment corresponds to the JSON schema field "prefix-development".
	PrefixDevelopment *string `json:"prefix-development,omitempty" yaml:"prefix-development,omitempty" mapstructure:"prefix-development,omitempty"`
}

type Condition struct {
	// DependencyName corresponds to the JSON schema field "dependency-name".
	DependencyName string `json:"dependency-name" yaml:"dependency-name" mapstructure:"dependency-name"`

	// Source corresponds to the JSON schema field "source".
	Source *string `json:"source,omitempty" yaml:"source,omitempty" mapstructure:"source,omitempty"`

	// UpdateTypes corresponds to the JSON schema field "update-types".
	UpdateTypes []string `json:"update-types,omitempty" yaml:"update-types,omitempty" mapstructure:"update-types,omitempty"`

	// UpdatedAt corresponds to the JSON schema field "updated-at".
	UpdatedAt *time.Time `json:"updated-at,omitempty" yaml:"updated-at,omitempty" mapstructure:"updated-at,omitempty"`

	// VersionRequirement corresponds to the JSON schema field "version-requirement".
	VersionRequirement *string `json:"version-requirement,omitempty" yaml:"version-requirement,omitempty" mapstructure:"version-requirement,omitempty"`
}

// Data that is passed to the updater
type DependabotJobSchemaJson struct {
	// Job corresponds to the JSON schema field "job".
	Job DependabotJobSchemaJsonJob `json:"job" yaml:"job" mapstructure:"job"`
}

type DependabotJobSchemaJsonJob struct {
	// AllowedUpdates corresponds to the JSON schema field "allowed-updates".
	AllowedUpdates []Allowed `json:"allowed-updates,omitempty" yaml:"allowed-updates,omitempty" mapstructure:"allowed-updates,omitempty"`

	// CommitMessageOptions corresponds to the JSON schema field
	// "commit-message-options".
	CommitMessageOptions *CommitOptions `json:"commit-message-options,omitempty" yaml:"commit-message-options,omitempty" mapstructure:"commit-message-options,omitempty"`

	// Cooldown corresponds to the JSON schema field "cooldown".
	Cooldown *UpdateCooldown `json:"cooldown,omitempty" yaml:"cooldown,omitempty" mapstructure:"cooldown,omitempty"`

	// CredentialsMetadata corresponds to the JSON schema field
	// "credentials-metadata".
	CredentialsMetadata []DependabotJobSchemaJsonJobCredentialsMetadataElem `json:"credentials-metadata,omitempty" yaml:"credentials-metadata,omitempty" mapstructure:"credentials-metadata,omitempty"`

	// Debug corresponds to the JSON schema field "debug".
	Debug *bool `json:"debug,omitempty" yaml:"debug,omitempty" mapstructure:"debug,omitempty"`

	// Dependencies corresponds to the JSON schema field "dependencies".
	Dependencies []string `json:"dependencies,omitempty" yaml:"dependencies,omitempty" mapstructure:"dependencies,omitempty"`

	// DependencyGroupToRefresh corresponds to the JSON schema field
	// "dependency-group-to-refresh".
	DependencyGroupToRefresh *string `json:"dependency-group-to-refresh,omitempty" yaml:"dependency-group-to-refresh,omitempty" mapstructure:"dependency-group-to-refresh,omitempty"`

	// DependencyGroups corresponds to the JSON schema field "dependency-groups".
	DependencyGroups []Group `json:"dependency-groups,omitempty" yaml:"dependency-groups,omitempty" mapstructure:"dependency-groups,omitempty"`

	// ExistingGroupPullRequests corresponds to the JSON schema field
	// "existing-group-pull-requests".
	ExistingGroupPullRequests []ExistingGroupPR `json:"existing-group-pull-requests,omitempty" yaml:"existing-group-pull-requests,omitempty" mapstructure:"existing-group-pull-requests,omitempty"`

	// ExistingPullRequests corresponds to the JSON schema field
	// "existing-pull-requests".
	ExistingPullRequests [][]ExistingPR `json:"existing-pull-requests,omitempty" yaml:"existing-pull-requests,omitempty" mapstructure:"existing-pull-requests,omitempty"`

	// Experiments corresponds to the JSON schema field "experiments".
	Experiments DependabotJobSchemaJsonJobExperiments `json:"experiments,omitempty" yaml:"experiments,omitempty" mapstructure:"experiments,omitempty"`

	// IgnoreConditions corresponds to the JSON schema field "ignore-conditions".
	IgnoreConditions []Condition `json:"ignore-conditions,omitempty" yaml:"ignore-conditions,omitempty" mapstructure:"ignore-conditions,omitempty"`

	// LockfileOnly corresponds to the JSON schema field "lockfile-only".
	LockfileOnly *bool `json:"lockfile-only,omitempty" yaml:"lockfile-only,omitempty" mapstructure:"lockfile-only,omitempty"`

	// MaxUpdaterRunTime corresponds to the JSON schema field "max-updater-run-time".
	MaxUpdaterRunTime *int `json:"max-updater-run-time,omitempty" yaml:"max-updater-run-time,omitempty" mapstructure:"max-updater-run-time,omitempty"`

	// Package manager being used
	PackageManager string `json:"package-manager" yaml:"package-manager" mapstructure:"package-manager"`

	// RejectExternalCode corresponds to the JSON schema field "reject-external-code".
	RejectExternalCode *bool `json:"reject-external-code,omitempty" yaml:"reject-external-code,omitempty" mapstructure:"reject-external-code,omitempty"`

	// RepoPrivate corresponds to the JSON schema field "repo-private".
	RepoPrivate *bool `json:"repo-private,omitempty" yaml:"repo-private,omitempty" mapstructure:"repo-private,omitempty"`

	// RequirementsUpdateStrategy corresponds to the JSON schema field
	// "requirements-update-strategy".
	RequirementsUpdateStrategy *string `json:"requirements-update-strategy,omitempty" yaml:"requirements-update-strategy,omitempty" mapstructure:"requirements-update-strategy,omitempty"`

	// SecurityAdvisories corresponds to the JSON schema field "security-advisories".
	SecurityAdvisories []Advisory `json:"security-advisories,omitempty" yaml:"security-advisories,omitempty" mapstructure:"security-advisories,omitempty"`

	// SecurityUpdatesOnly corresponds to the JSON schema field
	// "security-updates-only".
	SecurityUpdatesOnly *bool `json:"security-updates-only,omitempty" yaml:"security-updates-only,omitempty" mapstructure:"security-updates-only,omitempty"`

	// Source corresponds to the JSON schema field "source".
	Source Source `json:"source" yaml:"source" mapstructure:"source"`

	// UpdateSubdependencies corresponds to the JSON schema field
	// "update-subdependencies".
	UpdateSubdependencies *bool `json:"update-subdependencies,omitempty" yaml:"update-subdependencies,omitempty" mapstructure:"update-subdependencies,omitempty"`

	// UpdatingAPullRequest corresponds to the JSON schema field
	// "updating-a-pull-request".
	UpdatingAPullRequest *bool `json:"updating-a-pull-request,omitempty" yaml:"updating-a-pull-request,omitempty" mapstructure:"updating-a-pull-request,omitempty"`

	// VendorDependencies corresponds to the JSON schema field "vendor-dependencies".
	VendorDependencies *bool `json:"vendor-dependencies,omitempty" yaml:"vendor-dependencies,omitempty" mapstructure:"vendor-dependencies,omitempty"`
}

type DependabotJobSchemaJsonJobCredentialsMetadataElem map[string]interface{}

type DependabotJobSchemaJsonJobExperiments map[string]interface{}

type Dependency struct {
	// Directory corresponds to the JSON schema field "directory".
	Directory *string `json:"directory,omitempty" yaml:"directory,omitempty" mapstructure:"directory,omitempty"`

	// Name corresponds to the JSON schema field "name".
	Name string `json:"name" yaml:"name" mapstructure:"name"`

	// PreviousRequirements corresponds to the JSON schema field
	// "previous-requirements".
	PreviousRequirements []Requirement `json:"previous-requirements,omitempty" yaml:"previous-requirements,omitempty" mapstructure:"previous-requirements,omitempty"`

	// PreviousVersion corresponds to the JSON schema field "previous-version".
	PreviousVersion *string `json:"previous-version,omitempty" yaml:"previous-version,omitempty" mapstructure:"previous-version,omitempty"`

	// Removed corresponds to the JSON schema field "removed".
	Removed *bool `json:"removed,omitempty" yaml:"removed,omitempty" mapstructure:"removed,omitempty"`

	// Requirements corresponds to the JSON schema field "requirements".
	Requirements []Requirement `json:"requirements" yaml:"requirements" mapstructure:"requirements"`

	// Version corresponds to the JSON schema field "version".
	Version string `json:"version" yaml:"version" mapstructure:"version"`
}

type ExistingGroupPR struct {
	// Dependencies corresponds to the JSON schema field "dependencies".
	Dependencies []ExistingPR `json:"dependencies" yaml:"dependencies" mapstructure:"dependencies"`

	// DependencyGroupName corresponds to the JSON schema field
	// "dependency-group-name".
	DependencyGroupName string `json:"dependency-group-name" yaml:"dependency-group-name" mapstructure:"dependency-group-name"`
}

type ExistingPR struct {
	// DependencyName corresponds to the JSON schema field "dependency-name".
	DependencyName string `json:"dependency-name" yaml:"dependency-name" mapstructure:"dependency-name"`

	// DependencyVersion corresponds to the JSON schema field "dependency-version".
	DependencyVersion string `json:"dependency-version" yaml:"dependency-version" mapstructure:"dependency-version"`

	// Directory corresponds to the JSON schema field "directory".
	Directory *string `json:"directory,omitempty" yaml:"directory,omitempty" mapstructure:"directory,omitempty"`
}

type Group struct {
	// AppliesTo corresponds to the JSON schema field "applies-to".
	AppliesTo *string `json:"applies-to,omitempty" yaml:"applies-to,omitempty" mapstructure:"applies-to,omitempty"`

	// Name corresponds to the JSON schema field "name".
	Name *string `json:"name,omitempty" yaml:"name,omitempty" mapstructure:"name,omitempty"`

	// Rules corresponds to the JSON schema field "rules".
	Rules GroupRules `json:"rules,omitempty" yaml:"rules,omitempty" mapstructure:"rules,omitempty"`
}

type GroupRules map[string]interface{}

type Requirement struct {
	// File corresponds to the JSON schema field "file".
	File string `json:"file" yaml:"file" mapstructure:"file"`

	// Groups corresponds to the JSON schema field "groups".
	Groups []interface{} `json:"groups" yaml:"groups" mapstructure:"groups"`

	// Metadata corresponds to the JSON schema field "metadata".
	Metadata RequirementMetadata `json:"metadata,omitempty" yaml:"metadata,omitempty" mapstructure:"metadata,omitempty"`

	// PreviousVersion corresponds to the JSON schema field "previous-version".
	PreviousVersion *string `json:"previous-version,omitempty" yaml:"previous-version,omitempty" mapstructure:"previous-version,omitempty"`

	// Requirement corresponds to the JSON schema field "requirement".
	Requirement string `json:"requirement" yaml:"requirement" mapstructure:"requirement"`

	// Source corresponds to the JSON schema field "source".
	Source RequirementSource `json:"source,omitempty" yaml:"source,omitempty" mapstructure:"source,omitempty"`

	// Version corresponds to the JSON schema field "version".
	Version *string `json:"version,omitempty" yaml:"version,omitempty" mapstructure:"version,omitempty"`
}

type RequirementMetadata map[string]interface{}

type RequirementSource map[string]interface{}

type Source struct {
	// ApiEndpoint corresponds to the JSON schema field "api-endpoint".
	ApiEndpoint *string `json:"api-endpoint,omitempty" yaml:"api-endpoint,omitempty" mapstructure:"api-endpoint,omitempty"`

	// Branch corresponds to the JSON schema field "branch".
	Branch *string `json:"branch,omitempty" yaml:"branch,omitempty" mapstructure:"branch,omitempty"`

	// Commit corresponds to the JSON schema field "commit".
	Commit *string `json:"commit,omitempty" yaml:"commit,omitempty" mapstructure:"commit,omitempty"`

	// Directories corresponds to the JSON schema field "directories".
	Directories []string `json:"directories,omitempty" yaml:"directories,omitempty" mapstructure:"directories,omitempty"`

	// Directory corresponds to the JSON schema field "directory".
	Directory *string `json:"directory,omitempty" yaml:"directory,omitempty" mapstructure:"directory,omitempty"`

	// Hostname corresponds to the JSON schema field "hostname".
	Hostname *string `json:"hostname,omitempty" yaml:"hostname,omitempty" mapstructure:"hostname,omitempty"`

	// Provider corresponds to the JSON schema field "provider".
	Provider SourceProvider `json:"provider" yaml:"provider" mapstructure:"provider"`

	// Repo corresponds to the JSON schema field "repo".
	Repo string `json:"repo" yaml:"repo" mapstructure:"repo"`
}

type SourceProvider string

const SourceProviderAzure SourceProvider = "azure"
const SourceProviderBitbucket SourceProvider = "bitbucket"
const SourceProviderCodecommit SourceProvider = "codecommit"
const SourceProviderGithub SourceProvider = "github"
const SourceProviderGitlab SourceProvider = "gitlab"

type UpdateCooldown struct {
	// DefaultDays corresponds to the JSON schema field "default-days".
	DefaultDays *int `json:"default-days,omitempty" yaml:"default-days,omitempty" mapstructure:"default-days,omitempty"`

	// Exclude corresponds to the JSON schema field "exclude".
	Exclude []string `json:"exclude,omitempty" yaml:"exclude,omitempty" mapstructure:"exclude,omitempty"`

	// Include corresponds to the JSON schema field "include".
	Include []string `json:"include,omitempty" yaml:"include,omitempty" mapstructure:"include,omitempty"`

	// SemverMajorDays corresponds to the JSON schema field "semver-major-days".
	SemverMajorDays *int `json:"semver-major-days,omitempty" yaml:"semver-major-days,omitempty" mapstructure:"semver-major-days,omitempty"`

	// SemverMinorDays corresponds to the JSON schema field "semver-minor-days".
	SemverMinorDays *int `json:"semver-minor-days,omitempty" yaml:"semver-minor-days,omitempty" mapstructure:"semver-minor-days,omitempty"`

	// SemverPatchDays corresponds to the JSON schema field "semver-patch-days".
	SemverPatchDays *int `json:"semver-patch-days,omitempty" yaml:"semver-patch-days,omitempty" mapstructure:"semver-patch-days,omitempty"`
}

I ran ./go-jsonschema ./internal/model/dependabot-job.schema.json --package model --only-models

@yeikel
Copy link

yeikel commented Jun 11, 2025

This is an interesting idea. Would it be possible to add it as part of the build process and always place it under internal/gen/ or similar?

I see no reason to keep the manual models if we can use generated models using the schema

Comment on lines +147 to +152
"directories": {
"type": "array",
"items": {
"type": "string"
}
},
Copy link

@yeikel yeikel Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"directories": {
"type": "array",
"items": {
"type": "string"
}
},
"directories": {
"type": "array",
"items": {
"type": "string"
},
"uniqueItems": true
},

@JamieMagee
Copy link
Contributor Author

@yeikel this was mainly a proof-of-concept, but it is one that seems to work and has support from the rest of my team. I haven't had time to follow-up on it. If I were to complete this work, I'd likely migrate this to a separate repository and publish packages for the schema types, because I'd need to produce 3 different packages: Ruby, NuGet, and Go.

@yeikel
Copy link

yeikel commented Jun 17, 2025

@yeikel this was mainly a proof-of-concept, but it is one that seems to work and has support from the rest of my team. I haven't had time to follow-up on it. If I were to complete this work, I'd likely migrate this to a separate repository and publish packages for the schema types, because I'd need to produce 3 different packages: Ruby, NuGet, and Go.

Awesome, thank you for the update. I'd love to see at some point when you get capacity

Comment on lines +102 to +107
"credentials-metadata": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
Copy link

@yeikel yeikel Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to define this with this schema or similar?

It is currently defined as an untyped blob, but ultimately core validates/parses this with a schema

        "credentials-metadata": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "type": {
                "type": "string",
                "enum": [
                  "git_source",
                  "maven_repository",
                  "docker_registry",
                  "npm_registry",
                  "composer_repository",
                  "nuget_feed",
                  "python_index",
                  "rubygems_server",
                  "terraform_registry",
                  "hex_organization"
                ]
              },
              "host": {
                "type": "string"
              },
              "url": {
                "type": "string"
              },
              "username": {
                "type": "string"
              },
              "password": {
                "type": "string"
              }
            }
          }
        },

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may also need to be part of a "credentials" block we can add?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants