Thoughts on GIT sub-modules

Working on a project, it may happen during the development, some members of the team, or a single developer, is working on a project, which sometimes is stored as different git repository, and used as sub-module in another one.

- main repo
  /deps/ - submodule B
           /deps/ - submodule A
           /src
           /include
         - submodule A
           /src
           /include
  /src
  /include

Now, after a while, when this “repository” is used extensively in other projects, it’s time to publish, I no longer need the sub-modules, since now is part of a single project.

Having several sub-modules, the simplest thing is to remove a sub-module keeping the files into the main repository.

How do I remove a submodule?

1. Delete the submodule from .gitmodules file.
2. Delete the submodule from .git/config file.
3. Run git rm --cached path_to_submodule
4. Remove the submodule's .git directory `rm -rf .git/modules/submodule`
5. git commit -m "Removed submodule <submodule>"
6. git add ./directory_submodule (and it's content)
7. git commit -m "Added file 

Easy isn’t it? Yet there is a problem, I removed the sub-module, I have the files into the project, but I don’t have any history. What happened into the sub-module, all its commit, any bug-fixes, features etc. etc. are in another repository, not here..

For GIT, I added it now, there is nothing else. But I wasn’t happy.

Now, If you wanted to import the patch with a squash commit, a single diff file patch is enough, but if you would like to keep track also of different commits, you need a patch file for every commit, doing a commit into the destination repository and so on till the merge request.

Get cod’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The right solution is to create a bundle, an archive used by git to transfer content of commits (author, metadata, reference to previous index etc.) without an active git server (I’m working from remote, I did a patch, I will transfer my patch to my colleague which can send back the new bundle… etc. etc.), but this is valid until we are working on the same project, it’s not valid across different repositories (here I want to merge a sub-module patch into mine, and they don’t have anything in common).

Git allows to merge branches not linked together, through the option “ — allow-unrelated-histories” that starting from a commit A and a commit B coming from two different histories create a commit C where everything is available, but, don’t try to go back into your commit history, because when you’ll jump to B or to A, there is nothing in common.

So, basically what did I?

git clone https://github.com/privateorganization/mainrepo
cd mainrepo
git remote add submoduleA_origin https://github.com/privateorganization/A
git fetch submoduleA_origin
git checkout -b submoduleA_master
mkdir libs/submoduleA
git mv submoduleA_fileA libssubmoduleA
.... repeat for every file
git commit -m "preparing to merge"
git checkout master
git merge submoduleA_master
git remote remove submoduleA_origin

This works, finally I had my main-repository, with the history of a sub-module, into the right directory (obviously, if A had files into common directory, I moved into the common directory and not in other places…), the history and whatever.

But, to be honest, I’m not satisfied. Every-time I jump back in the log, I cannot switch to a specific commit which contains a full snapshot of the content of the main repository plus the sub-modules at the specific time.. because they don’t belong each other.

The only things I have are a lot of commits, many of them completely useless at this point, lots of backups commit, because a squash commit was not done, a dozen totally different branches that come off from completely random points of my history and a headache.

b45f48e (HEAD -> master, origin/master, origin/HEAD) fix: add allocate check for replace_item_in_object (#675)
a6424b8 feat: add cJSON_SetBoolValue and test (#639)
3cecc40 docs: Fix README typo (#664)
2fc55f6 chore: add CIFuzz integration (#437)
c7025b0 chore: ignore all .dylib files (#628)
e7ebe77 fix: 'cjson_utils-static' target not exist(#625)
61eb84d add an option for ENABLE_CJSON_VERSION_SO in CMakeLists.txt (#534)
d321fa9 fix: print int without decimal places (#630)
203a0de chore: ignore *.lst/*.lss file (#623)
c77a688 build: Bump cmake version and use new version syntax (#587)
e5dbaee docs: update comment (#622)
189dcde fix: add cmake_policy to CMakeLists.txt (#613)
b9eff8b fix: for issue #569, now use the guard to turn it off (#617)
f50dafc fix: potential memory leak in merge_patch() (#611)
d348621 (tag: v1.7.15) chore: update version and changelog (#610)
744e473 fix: remove redundant condition (#605)
7795249 Typos found by codespell (#607)
324a6ac Update .gitattributes (#544)
6ea4c01 Fix potential core dumped for strrchr (#546)
9226e4e Remove always true condition in cJSON.c (#539)
7b66457 Fix null pointer crash, closes #536 (#538)
4100379 docs: fix simple typo, transfering -> transferring (#527)

This log is extracted from cJSON, where each commit is linked to a specific issue, my log is totally sucks. I want to clean it, to collect all the commit from every sub-module, identify merge commit from different branch, creating a squash, and clean it.

I’m getting the idea of writing a tool/script which access at the same time to all repository (main, A, B, C and D), extracting every single commit, rearrange in time order, and rebuild every single bundle file (or the equivalent of a squash commit) and clean everything. In other words, If I’ll go to first commit, the other commits will be commits of sub-module reorganized from the first moment until ^HEAD of every single branch, so that I’ll have a new commit-hash when I need to create a branch (actually present only in a sub-module) where then I can go to apply branch commit…