SourceTree

Using submodules and subrepositories

By on February 1, 2012

A headline feature of SourceTree 1.3 is the support for submodules (in Git) and subrepositories (in Mercurial). The terms may be different, but they refer to the same concept; that of nesting other repositories within the folder structure of your own repository. The most common reason for wanting to do this is that your project has dependencies on other code bases (libraries for example), and you want to track those from their original sources rather than duplicating the files within your own repository.

For the sake of brevity from here on I’ll use the term ‘submodule’ to mean ‘subrepository’ as well, unless I’m talking about a Mercurial-specific feature.

Adding a submodule to your project

Adding a new submodule to your project is simple, just right-click on a blank area of the sidebar and select ‘New Submodule’ (or select it from the Repository menu).

You’ll then be prompted to provide a source URL to clone the contents from, and the path within the current repository that this submodule will reside. Once the submodule has cloned, you’ll see it appear in the sidebar like this:

This tells you that your submodule is located in dependencies/sub1 and is currently on the ‘master’ branch. If you wanted to see more detail about the submodule, just double-click on it to open it in its own repository window, from which you can, if you like, make changes to it just like any other repository. Submodule entries can also have annotations to let you know if there are uncommitted changes in the submodule (ellipsis annotation, i.e. ‘…’), or incoming / outgoing changes (up/down arrow).

In this case you’ve just added the submodule, but it’s not actually committed yet. You’ll see in the file status view that a couple of entries have been staged:

In Mercurial, there will be a “.hgsub” file instead but the principle is the same. These uncommitted changes represent the addition of the submodule to your repository, and you must commit & push them as you would do with any other change. Once you’ve done this, anyone cloning the repository will also get the submodule, at the same commit.

Typical submodule tasks

Submodules are likely to be more static than your own main repository, but there are still several tasks related to them that you may want to do:

Change the submodule commit which your repository is tracking

Submodules are pinned at a given commit, so that you always get the expected version of the submodule contents. If there are new changes in the submodule which you want to use in your repository, all you have to do is open the submodule (double-click) and checkout/update to a different commit, either via pull, merge or just manually checking a commit out – just as you would do normally with any other repository. Once you have done this, return to the parent repository and you will see an uncommitted change on the submodule, which will show in the diff the change in tracked commit:

Again, you commit & push this change just like any other, after which point the submodule contents will be pinned at this new commit.

Commit / push changes to a submodule together with parent repository

If you have write access to the upstream submodule source (and if not, maybe you might fork it and change the submodule source, see below), you may sometimes update files in the submodule, and want to commit / push those changes at the same time as you commit / push changes to your parent repository.

Mercurial actually requires this, by default it automatically commits subrepository changes when you commit the parent, using the same commit message. However, that’s often not what you want – there’s a good chance you’ll want a different commit message for the submodule change. Luckily, SourceTree makes that really easy – when you try to commit the parent, if there are uncommitted changes in submodules you’ll get a prompt telling you so, like this:

From this dialog you can click the ‘Commit’ button next to the submodule and commit those changes with a message of your choosing, separate to the parent repo commit message. In Git, the ‘Skip’ button at the bottom is enabled so you can just leave the changes uncommitted if you like, but Mercurial doesn’t allow this.

Change the source URL for a submodule

A submodule definition includes the URL which people cloning the parent repository will retrieve the submodule content from. Sometimes, you might want to change this – a common example is if you were using a 3rd party library which you had only read-access to, but later decided to fork it and make your own changes. In this case, you’ll want to change the source URL to your fork for all future clones / updates. In SourceTree this is easy, just right-click on the submodule and select “Change Source URL”. You’ll need to commit and push the change as normal, but after that your fork will be the source for the submodule.

Summary

This has been a quick introduction to the submodule and subrepository support in SourceTree 1.3. It’s designed to make your life easier when managing projects with their dependencies and other project groupings, we hope you like it!

24 Comments

  • Alex Peterson
    Posted February 1, 2012 at 10:37 pm | Permalink

    hawtness!!!

  • Gustavo
    Posted February 3, 2012 at 2:57 am | Permalink

    I was excited about as I have a lot of projects with submodules. But my existing project’s submodules do not come up on SourceTree. Any reason for this?

    • Anonymous
      Posted February 3, 2012 at 10:44 am | Permalink

      Make sure you click ‘Show’ next to the Submodules section. Otherwise, are you using 1.3.0 (App Store version) and git 1.7.8 on the command line? Unfortunately git changed the way they clone submodules in 1.7.8 which was just after 1.3.0 was packaged. 1.3.1 copes with this, you can get it direct from the website if you don’t want to wait for 1.3.1 to get through App Store review.

      • Gustavo
        Posted February 3, 2012 at 3:05 pm | Permalink

        Upgraded to git 1.7.8.4. No dice. Downloaded st 3.1.1. No dice. My problem is that my submodules were added by an earlier version of git (no entries in .git/config). Best solution: remove submodules (http://stackoverflow.com/questions/1260748/how-do-i-remove-a-git-submodule) and add again using SourceTree. Now I got it working! Thanks a lot. And thanks for this great app, imho beats the crap out of every other paid GUI for git.

  • Jerry
    Posted March 20, 2012 at 4:02 am | Permalink

    I was just trying out this feature and it looks like SourceTree is even updating the submodules (on a Git repository) to the correct revision when switching branches on the superproject (and probably for pulling in updates to the superproject). Is this true? If so, you should really be making more noise about this, as it’s a huge leap forward over the command-line Git client (which still seems to require a manual submodule update step, as far as I’m aware)!

    The most recent Git version seems to be better about warning you when your submodules are out of sync, but having to do a second step to update them every time is still a nuisance. If SourceTree takes care of that intelligently, that will make working with submodules so much easier!

  • Jon
    Posted August 16, 2012 at 10:15 am | Permalink

    I’m trying this out with SourceTree 1.5.3 and when I use Add Submodule and supply the URL (or file path, I’ve tried both on remote and local repos) All I get is the .gitmodules file. The submodule is not being cloned and I get an empty new sub-folder in my current repository. Am I missing something?

    • Anonymous
      Posted August 17, 2012 at 2:19 am | Permalink

      Hmm, no the init call to the submodule asks it to clone too. Also, if you double-click on the submodule on the sidebar after you’ve added it, if the submodule hasn’t been cloned already then SourceTree will try to do so.

      • Jon
        Posted August 17, 2012 at 2:48 am | Permalink

        Thanks for getting back to me.
        When I click on ‘Add Submodule…’ a dialog pops up that asks for the source repository and the target relative folder. I’ve tried adding a repository that is remotely hosted and also a local file-based repository. When the dialog completes, I don’t see anything under the ‘Submodules’ twisty on the left and all I get is a .gitmodules in my ‘Uncommitted changes’ in the repository. The other repository is not being cloned into the target relative folder – all that’s in there is a .git file.
        I am using the GitFlow stuff, is there any other setup I should have done before attempting to include the submodule?

        Thanks

        • Anonymous
          Posted August 17, 2012 at 2:56 am | Permalink

          No, that should be fine. Are you sure there’s nothing under Submodules – please note that in Lion+ there’s no disclosure triangle to the left of top-level items in a ‘Source List’ (iTunes-like) tree, you have to hover your mouse over it and a ‘Show’ or ‘Hide’ link appears to the right – clicking Show expands it (I really don’t like this change).

          If there definitely isn’t, try enabling the ‘Always show full output’ option in Preferences so that the Add Submodule dialog doesn’t close on success, then you can see if there’s anything displayed there which might hint at the cause. If there were errors the dialog wouldn’t close usually, but maybe worth a shot.

          If none of this helps, you can take this to https://support.atlassian.com if you want to discuss in more detail and provide private links etc for us to investigate.

          • Jon
            Posted August 17, 2012 at 4:21 am | Permalink

            Thanks for this. I’ve resolved the problem.
            What was not clear to me was that I need to just type the name of the target relative folder in the Add Submodule dialog. Adding the full messages was excellent, BTW!

            What was going on?
            Instead of just typing a target local folder name, I used the […] button to select a folder and as there was not one, I used the ‘Create Folder’ option to create it.
            SourceTree didn’t like this as now there was a new folder in the repo that hadn’t been added to the tracked list. Interestingly, I couldn’t add an empty folder to the working set, so I dropped a test file into the folder and tried again. Full console output showed that this wouldn’t work as you can’t put a submodule into a tracked folder. Makes sense.
            At this point I realised that I should try just typing the name of a folder into the ‘Local Relative Path’ field and this worked a treat. Note that Git seems to ignore case in file names.

            Thanks again for your speedy response!

          • Jon
            Posted August 17, 2012 at 6:40 am | Permalink

            Edit on the above. I was incorrect about using […] to create a new local folder for the submodule. You CAN do that too.
            Some further investigation suggests that some failed previous attempts to include a submodule may have left some artefacts in the .git/modules folder that were messing things up.

            Again, the ‘show full console output’ option was extremely useful.

            Thanks again

  • Jason
    Posted December 23, 2012 at 11:46 am | Permalink

    When I commit, I want all my subrepos committed. I actually prefer the automated commits. Is there anyway to have SourceTree auto-commit?

  • Jeudí Prando Araújo
    Posted April 3, 2013 at 8:45 am | Permalink

    no show option in sidebar for submodule in sourcetree app
    and in my repo exists configuration of submodule running perfect in git bash
    how to show in sidebar option for submodule?

    • Anonymous
      Posted April 4, 2013 at 6:38 am | Permalink

      You’re probably using the Windows version which currently doesn’t have full submodule support. I’m working on it now, it will be in an update soon.

  • Django
    Posted July 5, 2013 at 2:49 pm | Permalink

    Hi. I am getting this error when i try to upload from a mac or PC. (Mac wording slightly different). fatal: reference is not a tree: 146edf3b99e599c568855df2cc0702b52829edc5

    I see things like this http://stackoverflow.com/questions/2155887/git-submodule-head-reference-is-not-a-tree-error
    But wonder if there is a way to fix it within source tree?
    Thanks in advance for any help.

    • Anonymous
      Posted July 8, 2013 at 2:34 am | Permalink

      You can open the submodule in its own window by double-clicking the submodule entry in the sidebar. Then checkout a valid point in the history of that submodule repository (wherever makes sense, double-clicking a log row for example). Then return to the parent repository, and you should see that the commit you’re tracking on the submodule has changed. Commit that change, and that will record that new submodule commit as the one you’re now tracking from the parent, which shouldn’t suffer from the ‘reference is not a tree’ problem, because you checked out a valid one.

  • Matthias
    Posted December 23, 2013 at 2:54 am | Permalink

    Hi, when I add a submodule using SourceTree 1.8.0.3 on a mavericks mac it clones and produces .gitmodules and changes the .get/config, but the submodule does not show up in the side bar.
    When pushing this repo to reomote and cloning it into another repo the submodule folder remains empty.

    Is this a bug or is it my mistake somewhere?

    ciao Matthias

    • Matthias
      Posted December 23, 2013 at 2:55 am | Permalink

      btw: pressing hide and show does NOT solve this..

  • Nik Mikros
    Posted March 26, 2014 at 2:42 pm | Permalink

    I am getting the following error when I try to add a submodule to a local folder child. I made two dummy repos on the server child and parent the way I normally do, I can push and pull from each without a problem and I am using version 1.8.1 on the mac.

    git -c diff.mnemonicprefix=false -c core.quotepath=false -c credential.helper=sourcetree submodule add -f ssh://nik@xxx.xxx.net/Volumes/data/repos/test/child.git child

    fatal: You are on a branch yet to be born

    Unable to checkout submodule ‘child’

    Reactivating local git directory for submodule ‘child’.

    Completed with errors, see above

    • Anonymous
      Posted March 27, 2014 at 2:50 am | Permalink

      Because submodules are tracked by commit, you need at least one commit on both ends of the relationship before it’ll work. When you say you created dummy repos, do they have at least one commit?

      • Anonymous
        Posted March 27, 2014 at 7:39 am | Permalink

        I thought of that too, they each have at least one commit. I am running the repo off a mac server, could that be a problem?

  • thinsoldier
    Posted August 2, 2015 at 5:21 pm | Permalink

    This is broken if your repositories are on BitBucket.