Mac App Store: Sandboxing Update

A few months ago, I discussed how new sandboxing requirements have prompted us to move away from  the Mac App Store for future releases. Since that blog post, a few things happened:

1. The deadline was delayed to 1st June, giving everyone some extra time
2. Apple made some changes to OS X to allow more behaviours to be supported within the sandbox
3. Apple decided that they would still allow bugfix updates to non-sandboxed apps that were already available in the Mac App Store prior to 1st June

All these moves were welcome, and we thank Apple for making them. We have subsequently been able to publish SourceTree 1.4 to the App Store. We even had an expedited bug fix approved after the 1st June deadline, which was very useful.

Going forward with future releases, however, the changes that have been made to the sandbox still do not quite address all of the issues we have with it. While we could work around them, it would downgrade the user experience, which has always been a red line for us. We also have to consider the fact that the main alternatives to SourceTree are not distributed on the Mac App Store and are therefore not constrained by these rules.

Therefore our position has not materially changed since the original decision: SourceTree 1.5 onwards will only be distributed via sourcetreeapp.com. We advise all users on the Mac App Store to migrate to the direct download version, either now or when 1.5 is released, so you can benefit from the awesome new stuff we have in store for you.

[This update is just to clarify the questions we’ve had from people who have seen the recent 1.4 updates on the Mac App Store. We have already started using the new Developer ID code signatures since 1.4.4, which means the direct download version is compatible with Gatekeeper on Mountain Lion — Apple’s recommended method to distribute apps outside the Mac App Store.]

Update December 2012:

We decided to speculatively submit further 1.5 updates to the Mac App Store anyway, and so far, reviewers have allowed the updates to be approved. Given the official line from Apple that only bugfix releases will be allowed on the Mac App Store without sandboxing, we cannot guarantee that further updates will be accepted. We will continue to submit them as a convenience for our users who still wish to use the Mac App Store version, for as long as they continue to be approved, but must warn you that this situation may not continue indefinitely – we’re basically at Apple’s mercy here. We continue to recommend using the direct version from sourcetreeapp.com which receives updates much faster (review delays on MAS are running at several weeks) and is also able to support some features which MAS does not allow.

Update January 2013:

Apple has now stopped accepting updates to SourceTree in the Mac App Store, as we expected, although a little later than we thought. Therefore SourceTree 1.5.6 will be the last version available on the Mac App Store, you should switch to the direct version from http://sourcetreeapp.com if you want future updates (and why wouldn’t you? 😉 ).

Say Hi at WWDC!

From June 11 – June 15, SourceTree creator Steve Streeting will be at Apple’s Worldwide Developers Conference.

Steve will be taking advantage of his first WWDC and attending as many talks as he can, and he’d love to meet you! Get your questions answered, score a free SourceTree t-shirt, or just share some SourceTree love.

Don’t have a WWDC ticket? Can’t find him at the event? No worries! Steve will be around for a couple of the WWDC Parties during the week. Catch him at:

• ★☆☆☆☆ at Atlassian on Monday at 6 PM
• The Bitbucket and SourceTree drinkup at Bloodhound on Tuesday at 6 PM.
• The WWDC Bash at Yerba Buena Gardens on Thursday at 6 PM

Look forward to seeing you!

Draft your commit messages in SourceTree 1.4

Another useful new feature introduced in SourceTree 1.4 is the ability to draft your commit messages as you work, potentially long before you hit the commit button. This feature lets you jot down any information you want to include in the commit message later, without having to remember it!

To start, click the new button at the bottom left of the repository window or press the keyboard shortcut ⇧⌘D.

In the pop-up window, draft the contents of your next commit message:

When you’re finished, just press Escape or click somewhere outside the pop-up. The draft commit is saved along with your repository, so it will persist even if you close SourceTree. You can refine the message whenever you like, and of course when you finally do open the commit dialog, your commit message will be automatically populated with your draft, to be committed as-is or refined one last time.

Draft commit messages were added in 1.4 to help streamline your work process, we hope you like this feature!

Bookmarks Window: Reloaded

SourceTree’s bookmarks window has always been one of its defining features. We focused on this feature for SourceTree 1.4 to give users a considerably enhanced view that offers key information and an improved experience. Let’s dive in and see what’s changed.

New Visual Style

The most obvious change we’ve made is the new visual style. We’ve redesigned the UI to make it easier on the eyes. After compacting the bookmarks window vertically, more projects can be seen at once. See the current branch that your repository is on, and — if necessary — rename bookmarks by simply pressing Return, as you would in Finder. We’ve also improved the way the Bookmarks Window animates tree expansions and dragging/dropping to make the entire experience much more intuitive.

Live Searching

Having a hard time finding a repository? Live search is your friend. It doesn’t matter where in the bookmarks tree the repository you’re looking for is. Just start typing in the search box and SourceTree will instantly drill down and find it.

Quick Re-sorting

If your bookmarks window is becoming a little too cluttered, you can sort your repositories alphabetically or by version control system. Right-click on a folder and select ‘Sort by Name’ or ‘Sort by Type’ to reorganize your bookmarks in a snap.

Actions Menu

Save time and access repository functions right from the bookmarks window with Action Menus. Click the gear to jump to the repository folder in Finder or launch a Custom Action you’ve defined.

Create New Repositories with Drag & Drop

Previously, only two things could be dragged & dropped into the bookmarks window: your existing Mercurial or Git repositories from Finder and clone URLs from your browser. With SourceTree 1.4, you can now drop a plain source code folder into the window. SourceTree will save you some steps and offer you the option to turn this folder into a Mercurial or Git repository.

We hope you like the new bookmarks window. Try it out and improve your development process today!

SourceTree 1.4 is here!

SourceTree just took another step forward today with SourceTree 1.4. In this release, we have:

• Changed the UI of the Bookmarks window to improve navigation
• Implemented a clone button in Bitbucket and Stash for instant access to your source
• Created a setup wizard to help connect users with their repositories
• Listened to customer input and added advanced Mercurial support
• Implemented commit message drafting to save you time.
• And a whole lot more!

Please note: as at the time of writing 1.4 is still awaiting Mac App Store approval, therefore the only way to get the new version today is via the direct link above.

What’s new?

Bookmarks window: reloaded

The bookmarks window has been completely redesigned to improve its visual style and usability. Search your bookmarks in real time, quickly navigating to the projects you want to find regardless of where they are in the tree. Easily re-sort the contents by name or repository type, and get access to common functions without having to open the full repository window first. You can even invoke Custom Actions directly from the list.

Bitbucket and Atlassian Stash support

Enjoy instant access to your source code by cloning with SourceTree from any repository. Click the ‘SourceTree’ clone option in Bitbucket and Stash, Atlassian’s new enterprise Git repository management system. Your clone details will automatically be configured in SourceTree for you to instantly download source.

Setup Wizard

To welcome new users to SourceTree and get them up to speed, we now include a setup wizard which packages the steps most people want to perform the first time they open SourceTree. In one short swoop, SourceTree can:

• gather your user details
• configure your accounts on Bitbucket, GitHub, and Kiln
• search your local drive for repositories that you might want to bookmark.

Commit message drafts

Public beta of SourceTree 1.4 available now!

We’re approaching the release of the next major version of SourceTree, and as always we’d like to get a little feedback on the upcoming release from our pool of eager enthusiasts, before we flip the switch for everyone. You guys like new stuff, right? 🙂

SourceTree 1.4 has a loads of things we think you’ll love, including a revamped Bookmarks window with live searching, a new visual style and lots of other goodies, an upgrade for the uncommitted line on the log view, full support for Mercurial bookmarks, commit message drafting and a whole bunch of other things.

We’d love you to try it out and let us know what you think; random feedback is welcome at @sourcetree, and if you hit any problems please report them on JIRA. Hope you like the beta!

Download SourceTree 1.4 Beta

Between a rock and a hard place – our decision to abandon the Mac App Store

Update 22 Feb 2012: Apple have now extended the deadline to June 1st. Hopefully they’re realising the difficulties they’re creating for developers and will address some of the remaining concerns in this time.

On March 1st, Apple will change the rules of the Mac App Store to require all applications to run inside of a ‘sandbox’. Unfortunately, this will disallow important SourceTree functionality that was previously acceptable under store rules. Complying with the sandboxing rules would force us to change SourceTree in ways that would remove features, damage the usability of the app, and hurt our users; therefore, we will no longer submit SourceTree updates to the Mac App Store after March 1st, 2012. New updates will be available, for free, directly from sourcetreeapp.com (and via the in-app update). We will continue to monitor the situation in case Apple improve their sandboxing implementation or revise their rules. Note that we will still be signing SourceTree with our Apple developer certificate so SourceTree should work fine with the default settings of Mac OS X 10.8 Mountain Lion when it’s released.

For the full story of what forced us to take this disappointing decision, keep reading.

Background

From 1st March 2012, the Mac App Store will require all submitted applications to have application sandboxing enabled. Sandboxing runs the application in a controlled VM with little or no access to file, network or other resources unless they are specifically granted by the OS, which may be done by request of the application (if that’s provided), or via a privileged action dynamically granting permission, such as the Open File system dialog. The main aspects which impact SourceTree are as follows:

Sandboxed applications…

1. only have automatic file access to their ‘Container’, which is a chroot-like setup for just that single application, with local versions of the user home directory, preferences folders, and so on
2. can only access other file locations if the user gives explicit access via an ‘Open File’ dialog, a recently-used file list, the restoration of windows in Lion, or drag/drop
3. cannot even necessarily have knowledge of the path to the ‘real’ user home directory, only to the container version
4. cannot send Apple events to communicate with other applications

SourceTree impact

Sandboxing has a significant effect on SourceTree – the sandbox was designed mainly with a simple ‘document editing’ model in mind – ie that access to files would be granted by users explicitly opening them (thus granting explicit access via a user interaction, with the open file dialog being a privileged system component which cannot be altered by the application), and that the functional process is self-contained. However, not all applications work this way, SourceTree included.

Apple have notionally asked for developer feedback on this, and I and other developers have been raising ‘Radar’ issues on this for months. However, of the many issues I’ve raised since mid-2011, only a couple have been actually addressed (see the ‘Good News’ section below). The others sat in the queue for many months before finally receiving a dismissive ‘boilerplate’ response simply said ‘this is a known issue and we’re tracking it internally, please keep an eye on the release notes of future releases of OS X’. Obviously, this is not ideal. We’ve been hoping that this situation would be resolved before the deadline, but this is looking increasingly unlikely, and as such we feel it’s a good time to let you know what the position is for SourceTree.

Firstly, let’s talk about what the sandbox prevents us from doing in future.

File access issues

Users can’t type paths anymore

The only way that access can be granted to files & folders now is if the user browses to them, or drag/drops them. SourceTree currently offers the ability to type or copy/paste paths directly into text boxes, and also has the ability to derive sensible defaults which make the users life far easier.

For example, in the current direct-download version of SourceTree, if you drag/drop, paste or type a URL into SourceTree, it automatically derives a default clone destination for you (and bookmark name), based on the default project folder you’ve selected in your Preferences and the trailing part of the clone URL. This is really convenient and is often what you want – all you have to do is press ‘OK’.

Under the sandbox, you’re not allowed to default these values, nor can you allow the user to type a destination path into the text box. The user is forced to use the File Open panel, even if they would have been quite happy with a default, or have a path or prefix in their clipboard that they wanted to use. I know from experience that many users prefer this approach, but Apple’s sandbox says we have to take it away. In practice, this means redesigning the dialog because it feels very forced to do it this way. This is a total backwards step in UX in my opinion, but required by the model.

Losing File Access on Restart

Edit: literally just after this post was released, Apple officially added documentation for a new API to restore access to previously bookmarked file resources. They’re sure cutting it fine, less than 2 weeks before the sandbox becomes mandatory! This doesn’t change our decision yet, but if they continue to improve things we may be able to revisit this.

Under the sandbox, the file access an application was previously granted based on user behaviour is reset every time it starts up. Only documents that were open in document windows, and which are restored by Lion, are available automatically.

A key function of SourceTree is to keep track of all your repositories in one place, and to instantly show you what the status is. This works fine with the sandbox when you first add the repositories, but when you restart the application all that access is lost, unless the repositories were open in full document windows (on Lion). Clearly, because SourceTree allows users to bookmark an unlimited number of repositories, which the user doesn’t want to have permanently open in full document windows, this is not acceptable – chances are we’ll be denied access to 90% of repositories in the bookmarks at restart, making the bookmarks view – a major differentiating feature of SourceTree – completely useless.

Many other developers do this sort of thing in their applications, and this has been acknowledged by Apple in response to bug reports raised by myself and others, but has now disappeared into the impenetrable blackness of their internal tracking system. The consensus right now is that apps should claim the temporary exception to the file access rule, effectively asking for full permission to any file. I’ll talk about the implications of using temporary exceptions later on.

Home directory issues

Shared tool configuration will stop working

Currently when you use SourceTree, your SSH keys and agent configuration are automatically picked up, so access to remotes over SSH ‘just works’.

However, under the sandbox, the user home directory is ‘jailed’ inside the container, meaning that the standard SSH configuration is not available. All SSH access will fail unless the configuration is duplicated inside the container, and the SSH key loaded into the agent separately. Since the user home directory inside the container is only ever accessible when running SourceTree, none of the command-line tutorials for configuring SSH will work with SourceTree anymore, it will need its own, separate SSH configuration tool – another case of an unnecessary step being forced on the user.

You can’t even automate the initial process, say for example trying to copy the users existing setup (even though it would get out of sync if they changed it). This is because although a ‘migration’ option is made available for the sandbox, its only option is to move files into the container, it cannot copy them. So when I tried to use this for example, SSH configuration would work properly in SourceTree but it had deleted the original files in the real user home directory, something that would be extremely unpopular (understatement!).

The only real option is to force the user to import their keys. Helping them out with this is a bit tricky in itself, since all the standard Cocoa calls like NSHomeDirectory() point at the container, so we don’t know from there what their real home directory is, although some lower-level system calls seem to (currently) still expose this – I suspect this may be an oversight that will be corrected later.

The same issues exist with Mercurial, Git and Subversion. Configuration that the user has already performed won’t be automatically brought across, we’d have to prompt the user to configure everything manually, regardless of whether they’ve already configured the command line.

The short of it is that we can work around this, but the experience for the user is clunkier because they need to get involved more directly, and if they change their central configuration later they have to sync it up manually. The bottom line is that competing products that aren’t in the Mac App Store will give a better experience – if a user asks ‘how come <insert other DVCS client not on the Mac App Store> just picks up my configuration automatically but SourceTree doesn’t?’ – and I have no good answer to that other than ‘Apple made me do it’.

Apple Event Issues

Integration with Terminal etc prohibited

Sandboxed applications aren’t allowed to send Apple Events – this is because by doing so they could perform actions that they themselves are not permitted to do. However since Apple Events are the primary way to provide application integration – it’s how SourceTree launches Terminal / iTerm and navigates to the project folder for example – disallowing it removes some very useful functionality.

Right now, you can ask for a temporary exception for specified applications so there is a workaround. However, see the section later on regarding the implications of using temporary exceptions.

I have raised a bug with Apple to ask them to consider prompting the user to give permission for sending Apple Events between apps, in the same way that access to the Keychain is granted. Again, this has been closed as ‘yeah we know’, with no indication of whether it will actually be resolved or not.

Miscellaneous Issues

‘Open With’ is no longer allowed

When editing some files, such as the git/hg config files, SourceTree explicitly opens them with TextEdit rather than allowing the users file association. This is because I discovered that many users had weird associations on these files (presumably because they have no extension), so they might be opened in a Terminal instead for example.

Under the sandbox, you cannot ‘Open With’ anymore, it’s disallowed. You can only open files with their associated editor. This has other implications over and above the config files, it also means I can’t implement an explicit ‘Open With’ feature on the file list, which has been requested by users in the past.

Transfer of settings / licenses between App Store & Direct versions will fail

Currently if you downloaded SourceTree from the App Store, the license applies to the direct version and your settings are automatically kept in sync between the two, meaning it’s seamless to go back and forth between them. This is very useful if an App Store user wants to try out a beta version, or in the case where a hotfix is required.

Sandboxing does not allow this because access to the settings outside the container is prohibited. If a user tried to switch between the two systems, the settings/bookmarks would have to be recreated and they’d need to apply for a license separately. Not a major issue but it just feels a little shoddy.

First run of sandboxed application deletes previous configuration

The first time a user runs the sandboxed version from the app store, all the preferences are moved into the container and deleted from their original locations. This means the user can never go back to a previous build. Probably not serious, but worth noting.

Implications of Using Temporary Exceptions

Some of the above issues can be temporarily avoided by using exceptions that Apple make available – these are declarative entries in the applications property list which open up the sandbox more than the default. While this overcomes some issues in the short term, it has the following issues:

1. They don’t address the core issue, they just avoid it.
2. Apple have explicitly said they will take these exceptions away at some undefined point in the future. Whether they will create fully-fledged techniques to address the issue the exception was used to overcome, and whether those techniques will be appropriate or not, is a complete unknown.
3. Using the exceptions has to be justified at every application review. If you use an exception, the reviewers will pick it out and judge whether your rationale for defying the sandbox model is justified. In my experience, reviewers are not very consistent and I’ve had to lodge appeals several times (all successful luckily) because a reviewer has decided that a behaviour that has been approved in the past is now unacceptable. We can probably expect some delays and some conflict over our use of sandbox exceptions.

Therefore, even if we can ‘get by’ with the exceptions, having made all the sacrifices in usability terms as well, we may still find ourselves having difficulty getting updates through the App Store, and the landscape will continue to evolve.

Good news

Despite most of the issues I’ve raised with the sandboxing being closed without any satisfactory resolution, I did have one issue resolved that was a major problem for me – previously the sandbox would block termination of processes, even those that you’d launched from your application. This was a major problem for people clicking ‘Cancel’ in a clone window, because you couldn’t interrupt the clone task and it would continue running forever. Luckily, this has been fixed in 10.7.3.

Conclusion

Fundamentally, sandboxing is a good idea. Asking applications to be specific about what they need to do, and exposing that to the system and users for validation is a good idea for security.

The trouble is, the sandboxing implementation currently in place on Mac OS X Lion doesn’t allow for all the behaviours that real Mac applications do right now, behaviours which are not at all contentious, are approved in the Mac App Store already, and indeed are very much appreciated by users. Applications designed a certain way (mainly, those which edit documents on demand and don’t do much else) won’t have a problem since their behaviours are adequately catered for in the rights that can be applied for, but tools which perform more complex behaviour, particularly when that involves integrating with other apps and tools, do many things that simply aren’t catalogued in the sandbox. Also, forcing users to use a browse dialog for absolutely every file path is extremely clunky outside of the simplistic single document  editing use case – it might provide a nice way for the sandbox to grant file permission, but from a UX perspective it’s incredibly constraining.

Being on the Mac App Store has always required a bit of extra effort and some inconvenience for developers, but it’s been worth it – the Mac App Store is a great distribution platform which users love, and any difficulty and inconvenience with the platform has been limited to us, the developers, barring making people wait a bit longer for updates. To comply with the sandbox as it’s currently specified, however, requires making SourceTree a worse experience for users – there are some temporary exceptions which mitigate this to a degree, but they’re still incomplete and liable to be taken away, with no indication of what the replacement might be (if anything).

We’ve agonised over this decision for months, but the bottom line is that the current sandboxing implementation is simply not compatible with a version of SourceTree that we’d be happy to release. We care a great deal about the experience our users have, and being forced to release what would be, in our view, a downgrade of the app is just not something we want to do.  For this reason, we are not currently planning on submitting any more updates to the Mac App Store after 1st March. Our understanding is that the latest version at that time will remain in the Mac App Store for people who still wish to use it, but beyond that updates will be made only through direct download from the website, at least until something changes.

Our sincere hope is that Apple will address the shortcomings of the sandboxing implementation in the future, and allow us to request the kind of behaviours we need to maintain the features that SourceTree users know & love. Above all, we’d love Apple to talk to developers more openly about this, and give clearer guidance on how these issues are going to be dealt with.  Judging by talk in the community, we’re far from the only developers wrestling with these kinds of problems.

Related reading:

• Sandboxing divides Mac App Store developers (ZDNet)
• Understanding The Debate Over Apple’s Mac App Store Sandbox (The Mac Observer)
• OS X Lion Sandboxing Is A Killjoy Destined To Ruin Our Mac Experience (Cult of Mac)
• It’s all about Sandboxing (iDeveloper Live, podcast)

Custom actions – more power to you

SourceTree 1.3’s new ‘Custom Actions’ feature lets you extend the range of actions you can perform from within the GUI, effectively adding your own commands. Maybe you have scripts that you’d like to call, or external tools that you’d like to use beyond what SourceTree already allows for – Custom Actions are the way to go.

As a simple example, let’s say you use TextMate for editing files in a project, and you wanted to assign a keyboard shortcut to open the root of the repository in TextMate? SourceTree doesn’t provide an in-built option for that, so let’s add it using Custom Actions.

Firstly, open the Preferences window, and select the ‘Custom Actions’ tab:

Click ‘Add’, and fill in the details as follows (to fill in the keyboard shortcut, just lick on the shortcut area and type a keyboard combination):

Notice how all I had to do was specify the location of the script I wanted to run, and then give it some parameters – in this case ‘$REPO’ which is the path to the repository.

After I click OK, an ‘Open in TextMate’ option will appear on the top-level Actions menu under the Custom Actions section:

Or, I can just press Shift-T to open my repository in TextMate whenever I want.

If you use the $FILE or $SHA parameters, then these will cause the action to appear on context menus associated with files or commits, and will pass that context to the command. So for example you could have a deployment script which took the $SHA parameter, then right-click on a commit and run your deployment script for that specific commit right from within SourceTree.

Custom Actions are a great new way to extend SourceTree to work with your own scripts and tools, and improve your productivity!

JIRA integration & other external project links

SourceTree 1.3 makes it easy to navigate to JIRA issues which are associated with your commits – and in fact to other supporting information like Crucible reviews or any predictable pattern in a commit message that you can identify.

Linking a repository to a JIRA project

If you’re committing code to resolve a JIRA issue, you’ll normally include the JIRA issue number in the commit message, perhaps something like this:

PROJECTX-101 Fixed bad encoding on UTF-16 characters

Wouldn’t it be awesome if when you looked at that commit message in the log, that ‘PROJECTX-101’ was a link to the JIRA issue? Well, it turns out that’s really easy in 1.3.

Simply click the ‘Settings’ button on the repository toolbar, select the Advanced tab, and look for the ‘Commit Text Replacements’ section:

Here you can see one I prepared earlier. To create it, I clicked ‘Add’ and entered these details:

All I needed to do was specify the project prefix (every JIRA project has one of these), and supply the base URL for my JIRA installation. With this in place, SourceTree will search for all instances of ‘PROJECTX-<number>’ in commit messages, and will hyperlink it to the issue itself, like this:

Clicking that link would open the issue at https://jira.myproject.com/browse/PROJECTX-101 in your default web browser.

Advanced usage – other types of links

It’s not only JIRA issues which can be linked. If you change the ‘Replacement Type’ box you can see there’s anther helper option for Crucible projects (which works much the same way as JIRA), but there is also an ‘Other’ option, which allows you to specify your own text replacements in terms of regular expressions. They could be links to other tracking systems, or indeed any other kind of replacement. Here’s an example:

You can specify a regular expression pattern to search for in the commit message, and what text should replace it. You can use capture clauses (parenthesis in the regex pattern, variable names in the ‘Replace With’ field) in order to extract details and reformat them. The keen-eyed among you may have noticed that the screenshot above actually represents the same JIRA link replacement, just in raw regex form – notice how it is looking for the issue pattern, and then replacing it with HTML to form a link. So you can do anything you need to do to translate commit plain text into HTML of your choosing.

We hope you enjoy this SourceTree 1.3 feature!

Using submodules and subrepositories

A headline feature of SourceTree 1.3 is the support for submodules (in Git) and subrepositories (in Mercurial). The terms may be different, but they refer to the same concept; that of nesting other repositories within the folder structure of your own repository. The most common reason for wanting to do this is that your project has dependencies on other code bases (libraries for example), and you want to track those from their original sources rather than duplicating the files within your own repository.

For the sake of brevity from here on I’ll use the term ‘submodule’ to mean ‘subrepository’ as well, unless I’m talking about a Mercurial-specific feature.

Adding a submodule to your project

Adding a new submodule to your project is simple, just right-click on a blank area of the sidebar and select ‘New Submodule’ (or select it from the Repository menu).

You’ll then be prompted to provide a source URL to clone the contents from, and the path within the current repository that this submodule will reside. Once the submodule has cloned, you’ll see it appear in the sidebar like this:

This tells you that your submodule is located in dependencies/sub1 and is currently on the ‘master’ branch. If you wanted to see more detail about the submodule, just double-click on it to open it in its own repository window, from which you can, if you like, make changes to it just like any other repository. Submodule entries can also have annotations to let you know if there are uncommitted changes in the submodule (ellipsis annotation, i.e. ‘…’), or incoming / outgoing changes (up/down arrow).

In this case you’ve just added the submodule, but it’s not actually committed yet. You’ll see in the file status view that a couple of entries have been staged:

In Mercurial, there will be a “.hgsub” file instead but the principle is the same. These uncommitted changes represent the addition of the submodule to your repository, and you must commit & push them as you would do with any other change. Once you’ve done this, anyone cloning the repository will also get the submodule, at the same commit.

Typical submodule tasks

Submodules are likely to be more static than your own main repository, but there are still several tasks related to them that you may want to do:

Change the submodule commit which your repository is tracking

Submodules are pinned at a given commit, so that you always get the expected version of the submodule contents. If there are new changes in the submodule which you want to use in your repository, all you have to do is open the submodule (double-click) and checkout/update to a different commit, either via pull, merge or just manually checking a commit out – just as you would do normally with any other repository. Once you have done this, return to the parent repository and you will see an uncommitted change on the submodule, which will show in the diff the change in tracked commit:

Again, you commit & push this change just like any other, after which point the submodule contents will be pinned at this new commit.

Commit / push changes to a submodule together with parent repository

If you have write access to the upstream submodule source (and if not, maybe you might fork it and change the submodule source, see below), you may sometimes update files in the submodule, and want to commit / push those changes at the same time as you commit / push changes to your parent repository.

Mercurial actually requires this, by default it automatically commits subrepository changes when you commit the parent, using the same commit message. However, that’s often not what you want – there’s a good chance you’ll want a different commit message for the submodule change. Luckily, SourceTree makes that really easy – when you try to commit the parent, if there are uncommitted changes in submodules you’ll get a prompt telling you so, like this:

From this dialog you can click the ‘Commit’ button next to the submodule and commit those changes with a message of your choosing, separate to the parent repo commit message. In Git, the ‘Skip’ button at the bottom is enabled so you can just leave the changes uncommitted if you like, but Mercurial doesn’t allow this.

Change the source URL for a submodule

A submodule definition includes the URL which people cloning the parent repository will retrieve the submodule content from. Sometimes, you might want to change this – a common example is if you were using a 3rd party library which you had only read-access to, but later decided to fork it and make your own changes. In this case, you’ll want to change the source URL to your fork for all future clones / updates. In SourceTree this is easy, just right-click on the submodule and select “Change Source URL”. You’ll need to commit and push the change as normal, but after that your fork will be the source for the submodule.

Summary

This has been a quick introduction to the submodule and subrepository support in SourceTree 1.3. It’s designed to make your life easier when managing projects with their dependencies and other project groupings, we hope you like it!