Skip to content
This repository was archived by the owner on Mar 13, 2025. It is now read-only.

Integrate enrich processor #56

Merged
merged 6 commits into from
Oct 22, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
- node 12.x
- npm 6.x
- docker
- elasticsearch 6.x
- elasticsearch 7.7

## Configuration

Expand Down Expand Up @@ -36,8 +36,16 @@ Configuration for the application is at config/default.js and config/production.
- UBAHN_DELETE_TOPIC: Kafka topic for delete message
- UBAHN_AGGREGATE_TOPIC: Kafka topic that is used to combine all create, update and delete message(s)
- ES.HOST: Elasticsearch host
- ES.API_VERSION: Elasticsearch API version
- ES.DOCUMENTS: Elasticsearch index, type and id mapping for resources.
- ATTRIBUTE_GROUP_PIPELINE_ID: The pipeline id for enrichment with attribute group. Default is `attributegroup-pipeline`
- SKILL_PROVIDER_PIPELINE_ID: The pipeline id for enrichment with skill provider. Default is `skillprovider-pipeline`
- USER_PIPELINE_ID: The pipeline id for enrichment of user details. Default is `user-pipeline`
- ATTRIBUTE_GROUP_ENRICH_POLICYNAME: The enrich policy for attribute group. Default is `attributegroup-policy`
- SKILL_PROVIDER_ENRICH_POLICYNAME: The enrich policy for skill provider. Default is `skillprovider-policy`
- ROLE_ENRICH_POLICYNAME: The enrich policy for role. Default is `role-policy`
- ACHIEVEMENT_PROVIDER_ENRICH_POLICYNAME: The enrich policy for achievement provider. Default is `achievementprovider-policy`
- SKILL_ENRICH_POLICYNAME: The enrich policy for skill. Default is `skill-policy`
- ATTRIBUTE_ENRICH_POLICYNAME: The enrich policy for skill. Default is `attribute-policy`

For `ES.DOCUMENTS` configuration, you will find multiple other configurations below it. Each has default values that you can override using the environment variables

Expand Down
29 changes: 19 additions & 10 deletions config/default.js
Original file line number Diff line number Diff line change
Expand Up @@ -50,41 +50,50 @@ module.exports = {

// ElasticSearch
ES: {
HOST: process.env.ES_HOST || 'localhost:9200',
API_VERSION: process.env.ES_API_VERSION || '7.4',
HOST: process.env.ES_HOST || 'http://localhost:9200',
// es mapping: _index, _type, _id
DOCUMENTS: {
achievementprovider: {
index: process.env.ACHIEVEMENT_PROVIDER_INDEX || 'achievement_provider',
type: '_doc'
type: '_doc',
enrichPolicyName: process.env.ACHIEVEMENT_PROVIDER_ENRICH_POLICYNAME || 'achievementprovider-policy'
},
attribute: {
index: process.env.ATTRIBUTE_INDEX || 'attribute',
type: '_doc'
type: '_doc',
enrichPolicyName: process.env.ATTRIBUTE_ENRICH_POLICYNAME || 'attribute-policy'
},
attributegroup: {
index: process.env.ATTRIBUTE_GROUP_INDEX || 'attribute_group',
type: '_doc'
type: '_doc',
pipelineId: process.env.ATTRIBUTE_GROUP_PIPELINE_ID || 'attributegroup-pipeline',
enrichPolicyName: process.env.ATTRIBUTE_GROUP_ENRICH_POLICYNAME || 'attributegroup-policy'
},
organization: {
index: process.env.ORGANIZATION_INDEX || 'organization',
type: '_doc'
type: '_doc',
enrichPolicyName: process.env.ORGANIZATION_ENRICH_POLICYNAME || 'organization-policy'
},
role: {
index: process.env.ROLE_INDEX || 'role',
type: '_doc'
type: '_doc',
enrichPolicyName: process.env.ROLE_ENRICH_POLICYNAME || 'role-policy'
},
skill: {
index: process.env.SKILL_INDEX || 'skill',
type: '_doc'
type: '_doc',
enrichPolicyName: process.env.SKILL_ENRICH_POLICYNAME || 'skill-policy'
},
skillprovider: {
index: process.env.SKILL_PROVIDER_INDEX || 'skill_provider',
type: '_doc'
type: '_doc',
pipelineId: process.env.SKILL_PROVIDER_PIPELINE_ID || 'skillprovider-pipeline',
enrichPolicyName: process.env.SKILL_PROVIDER_ENRICH_POLICYNAME || 'skillprovider-policy'
},
user: {
index: process.env.USER_INDEX || 'user',
type: '_doc'
type: '_doc',
pipelineId: process.env.USER_PIPELINE_ID || 'user-pipeline'
},
// sub resources under user
achievement: {
Expand Down
Loading