[strict yang] Add YANG based cfg_schema.h to schema.h #1008

wen587 · 2025-04-18T01:32:01Z

Why I did it

To enable YANG based CFG table generation, the ConfigDB related macro will be generated by YANG model

How I did it

Add gen_cfg_schema.py script to generate cfg_schema.h which will be included in schema.h

Work item tracking

Microsoft ADO: 31676863

How to verify it

make deb build

Which release branch to backport (provide reason below if selected)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

mssonicbld · 2025-04-18T01:32:08Z

/azp run

azure-pipelines · 2025-04-18T01:32:14Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

mssonicbld · 2025-04-18T01:46:49Z

/azp run

azure-pipelines · 2025-04-18T01:46:55Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

mssonicbld · 2025-04-18T02:04:15Z

/azp run

azure-pipelines · 2025-04-18T02:04:27Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-04-18T06:41:17Z

/azp run

azure-pipelines · 2025-04-18T06:41:23Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

mssonicbld · 2025-04-18T06:45:18Z

/azp run

azure-pipelines · 2025-04-18T06:45:30Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-04-18T06:51:45Z

/azp run

azure-pipelines · 2025-04-18T06:51:56Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-04-18T06:57:22Z

/azp run

azure-pipelines · 2025-04-18T06:57:33Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-04-21T01:05:19Z

/azp run

azure-pipelines · 2025-04-21T01:05:30Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-04-21T06:58:12Z

/azp run

azure-pipelines · 2025-04-21T06:58:25Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-04-21T10:18:42Z

/azp run

azure-pipelines · 2025-04-21T10:18:55Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-04-22T05:35:18Z

/azp run

azure-pipelines · 2025-04-22T05:35:29Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-04-22T05:56:54Z

/azp run

azure-pipelines · 2025-04-22T05:57:06Z

Azure Pipelines successfully started running 1 pipeline(s).

…-net#1052) This PR is a partial duplicate of sonic-net/sonic-dash-ha#84 Since crates/swss-common will be moved in this repo. why During investigating issue sonic-net#75, I created test case to verify zmq behaviour in handling no connection and connection loss. It is found that with the current swss-common zmq implementation with below attributes, we don't need to handle reconnect explicitly. PUSH/PULL model ZMQ_IMMEDIATE = 0 (default) ZMQ_SNDHWM = 10000 what this PR does add unit test for zmq late connect and reconnect for regression remove unneeded code for falling back to ProducerStateTable in test environment since zmq client won't fail if zmq server is not connected. update test_utils/README.md with missing argument to start redis in test environment

HLD: sonic-net/SONiC#2033

Moving SonicDbTable to sonic-dash-ha as part of sonic-net/sonic-dash-ha#82

azure-pipelines.yml

Copilot

Pull Request Overview

This PR enables YANG-based configuration table generation by introducing an automated script to generate CFG_* macro definitions from YANG models. The changes integrate the generation of cfg_schema.h into the build process and update the schema to use this dynamically generated header instead of hardcoded table definitions.

Adds gen_cfg_schema.py script that dynamically generates CFG_* table macros from YANG models
Integrates the generation into the build system through Makefile changes and CI pipeline updates
Replaces hardcoded CFG_* table definitions in schema.h with inclusion of the generated cfg_schema.h

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
gen_cfg_schema.py	New Python script that extracts table names from YANG models and generates CFG_* macro definitions
tests/test_gen_cfg_schema.py	Unit tests for the schema generation script
common/Makefile.am	Build system integration to generate cfg_schema.h before compilation
common/schema.h	Replaces hardcoded CFG_* macros with include of generated cfg_schema.h
pyext/swsscommon.i	Adds cfg_schema.h to SWIG interface for Python bindings
azure-pipelines.yml	Updates CI to install YANG dependencies for schema generation
.azure-pipelines/*.yml	Additional CI updates for YANG package installation

Comments suppressed due to low confidence (1)

tests/test_gen_cfg_schema.py:46

The test only covers the case where a key contains a hyphen, but doesn't test other potentially invalid characters for C macro names (e.g., spaces, dots, special symbols). Consider adding test cases for other invalid characters that could appear in YANG model names.

            # Check that memory-usage is commented out due to hyphen

gen_cfg_schema.py

.azure-pipelines/build-template.yml

.azure-pipelines/test-docker-sonic-vs-template.yml

mssonicbld · 2025-08-06T11:06:10Z

/azp run

azure-pipelines · 2025-08-06T11:06:23Z

Azure Pipelines successfully started running 1 pipeline(s).

yue-fred-gao · 2025-08-13T13:30:53Z

@wen587 , I tried to update swss-common submodule in sonic-buildimage with PR sonic-net/sonic-buildimage#23687. There is a compilation error

/usr/bin/python ./gen_cfg_schema.py -o common/cfg_schema.h
Traceback (most recent call last):
  File "./gen_cfg_schema.py", line 2, in <module>
    import sonic_yang
ImportError: No module named sonic_yang
make[3]: *** [Makefile:3503: common/cfg_schema.h] Error 1

Is there another dependent submodule that I need to add to sonic-buildimage?

wen587 · 2025-08-13T13:36:57Z

@wen587 , I tried to update swss-common submodule in sonic-buildimage with PR sonic-net/sonic-buildimage#23687. There is a compilation error
/usr/bin/python ./gen_cfg_schema.py -o common/cfg_schema.h
Traceback (most recent call last):
  File "./gen_cfg_schema.py", line 2, in <module>
    import sonic_yang
ImportError: No module named sonic_yang
make[3]: *** [Makefile:3503: common/cfg_schema.h] Error 1
Is there another dependent submodule that I need to add to sonic-buildimage?

Hi @yue-fred-gao , no . Investigating the compliation issue.

…net#1008)" This reverts commit 839ddf0.

#1067) This reverts commit 839ddf0.

wen587 · 2025-08-14T06:51:28Z

Hi @yue-fred-gao , pr reverted. Please go ahead update your submodule.

yue-fred-gao · 2025-08-14T13:58:00Z

Thanks for the quick solution!

…sonic-net#1008)" (sonic-net#1067)" This reverts commit dc05565.

Comparing to #1008, change to PYTHON3 for buster build failure. Improve the dynamic debian_version as an argument instead of static bookworm. Why I did it To enable YANG based CFG table generation, the ConfigDB related macro will be generated by YANG model How I did it Add gen_cfg_schema.py script to generate cfg_schema.h which will be included in schema.h Work item tracking Microsoft ADO: 31676863 How to verify it make deb build

wen587 force-pushed the libc6 branch from 01062f0 to 1259692 Compare April 18, 2025 06:41

wen587 changed the title ~~Libc6~~ Automatically generate CFG table by YANG May 20, 2025

wen587 changed the title ~~Automatically generate CFG table by YANG~~ [strict yang] Add YANG based cfg_schema.h to schema.h Jun 24, 2025

wen587 added 3 commits June 24, 2025 08:14

1. finish gen cfg_schema.h during build

0491f01

2. remove from make, add in deb build

601eb79

1.1 should add py early

667d54a

qiluo-msft and others added 6 commits August 5, 2025 12:58

[trim]: Add Packet Trimming Drop Counters DB schema (sonic-net#1055)

f827e73

HLD: sonic-net/SONiC#2033

[MSTP] Schema Support (sonic-net#961)

f071423

Move SonicDbTable to sonic-dash-ha. (sonic-net#1056)

56112c1

Moving SonicDbTable to sonic-dash-ha as part of sonic-net/sonic-dash-ha#82

add to python binding

0610bbd

fix configlict

9ba7131

qiluo-msft reviewed Aug 6, 2025

View reviewed changes

azure-pipelines.yml Outdated Show resolved Hide resolved

qiluo-msft reviewed Aug 6, 2025

View reviewed changes

azure-pipelines.yml Outdated Show resolved Hide resolved

qiluo-msft reviewed Aug 6, 2025

View reviewed changes

azure-pipelines.yml Outdated Show resolved Hide resolved

qiluo-msft reviewed Aug 6, 2025

View reviewed changes

azure-pipelines.yml Outdated Show resolved Hide resolved

qiluo-msft reviewed Aug 6, 2025

View reviewed changes

azure-pipelines.yml Outdated Show resolved Hide resolved

qiluo-msft reviewed Aug 6, 2025

View reviewed changes

azure-pipelines.yml Outdated Show resolved Hide resolved

qiluo-msft requested a review from Copilot August 6, 2025 00:24

Copilot AI reviewed Aug 6, 2025

View reviewed changes

gen_cfg_schema.py Show resolved Hide resolved

gen_cfg_schema.py Show resolved Hide resolved

qiluo-msft reviewed Aug 6, 2025

View reviewed changes

.azure-pipelines/build-template.yml Outdated Show resolved Hide resolved

qiluo-msft reviewed Aug 6, 2025

View reviewed changes

.azure-pipelines/test-docker-sonic-vs-template.yml Outdated Show resolved Hide resolved

resolve comment and specific install

6d17ff8

qiluo-msft approved these changes Aug 8, 2025

View reviewed changes

qiluo-msft merged commit 839ddf0 into sonic-net:master Aug 8, 2025
15 checks passed

wen587 added a commit to wen587/sonic-swss-common that referenced this pull request Aug 13, 2025

Revert "[strict yang] Add YANG based cfg_schema.h to schema.h (sonic-…

9b75af9

…net#1008)" This reverts commit 839ddf0.

qiluo-msft pushed a commit that referenced this pull request Aug 14, 2025

Revert "[strict yang] Add YANG based cfg_schema.h to schema.h (#1008)" (

dc05565

#1067) This reverts commit 839ddf0.

wen587 added a commit to wen587/sonic-swss-common that referenced this pull request Sep 4, 2025

Revert "Revert "[strict yang] Add YANG based cfg_schema.h to schema.h (…

1425c91

…sonic-net#1008)" (sonic-net#1067)" This reverts commit dc05565.

wen587 mentioned this pull request Sep 19, 2025

[strict yang] Add YANG based cfg_schema.h to schema.h #1074

Merged

6 tasks

[strict yang] Add YANG based cfg_schema.h to schema.h #1008

[strict yang] Add YANG based cfg_schema.h to schema.h #1008

Uh oh!

Conversation

wen587 commented Apr 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why I did it

How I did it

Work item tracking

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Uh oh!

mssonicbld commented Apr 18, 2025

Uh oh!

azure-pipelines bot commented Apr 18, 2025

Uh oh!

mssonicbld commented Apr 18, 2025

Uh oh!

azure-pipelines bot commented Apr 18, 2025

Uh oh!

mssonicbld commented Apr 18, 2025

Uh oh!

azure-pipelines bot commented Apr 18, 2025

Uh oh!

mssonicbld commented Apr 18, 2025

Uh oh!

azure-pipelines bot commented Apr 18, 2025

Uh oh!

mssonicbld commented Apr 18, 2025

Uh oh!

azure-pipelines bot commented Apr 18, 2025

Uh oh!

mssonicbld commented Apr 18, 2025

Uh oh!

azure-pipelines bot commented Apr 18, 2025

Uh oh!

mssonicbld commented Apr 18, 2025

Uh oh!

azure-pipelines bot commented Apr 18, 2025

Uh oh!

mssonicbld commented Apr 21, 2025

Uh oh!

azure-pipelines bot commented Apr 21, 2025

Uh oh!

mssonicbld commented Apr 21, 2025

Uh oh!

azure-pipelines bot commented Apr 21, 2025

Uh oh!

mssonicbld commented Apr 21, 2025

Uh oh!

azure-pipelines bot commented Apr 21, 2025

Uh oh!

mssonicbld commented Apr 22, 2025

Uh oh!

azure-pipelines bot commented Apr 22, 2025

Uh oh!

mssonicbld commented Apr 22, 2025

Uh oh!

azure-pipelines bot commented Apr 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mssonicbld commented Aug 6, 2025

Uh oh!

wen587 commented Apr 18, 2025 •

edited

Loading