Dependency Management with .NET – Doing it Right

The problem of dependency management is neither new nor original, it exists in all development platforms, and .NET is no different.
Let’s go through different solutions and see how they perform. I’ll list them here in no particular order.

Keeping dependencies in your source control
That’s a very popular solution, and for a reason. The benefits are obvious. Here are some of them:

• No setup. You already have your source control in place (hm, I hope you do!). Add bin directory, and you are fine.
• No learning curve. Developers are used to work with source control.
• Shared. The whole team getschangesand updates from the server as they occur.
• Enterprisy(in a good way). The software is proven, backed up, DRP is done.

Sounds good, doesn’t it? So, what’s wrong with it? Only one thing – source control systems are designed to control, well, sources. Assuch theyaren’t so great in controlling binaries. These are the shortcomings we all encountered during the years of Version Control System usage for dependencies:

• It isn’t a proxy. VCS can’t download the dependency you needfroma central repository when you need one. You need to manually download it and add it to VCS. The history starts from there – you just lost the link to the original file. So, you work hard, and on top of it lost information; this repeatitselffor each new dependency.
• Versioningmismatch. Source files areversionedby their content.VCSsknow how todiffthem and understand what changed. Binaries, on the other side, usuallyversionedby their name. FromVCS pointof view they are different entries, each one without any version history.
• Some very popular VCSs (like Subversion) can’t obliterate files. That means – once a file was added, itstayin the repository forever. That’s not a big issue for small source files, but can become quite a pain when it comes to obsolete large binaries.
• Source control knows how to search sources. And, of course, the most important type of search is by content. Searching for binaries is different: what matters there is the location, structure of the file name and, in case of archived artifact, the contents ofarchive.
• Thepermissionsscheme of VCSs is tailored forversioning来源(gain!). For example, there is no override permission. That’s because overriding sources is something we do all the time (that’s whatdiffis for in VCS) – it’s the same security level as, let’s say, adding a new source file. With binaries the situation is very different. While adding new binaries is fine, overriding released binary is something that shouldn’t be done, one should have a specialpermissionfor it.
• Distributed VCSs, awesome by themselves, are particularly unsuited for handling big binary files. When cloning a remote repository to yourmachineyouare bringing all the history of all the files in it. Now just think about all the huge binaries sitting there…

As you see, the conclusion is simple – we can do better. Let’s try something specialized for binaries.

GAC and WebGAC
Global Assembly Cache is, on contradictory to VCS,tailored for storing binaries. It understands versions, prevents conflicts, and generally does a good job being your local dependencies storage. The main problem with GAC is being local, which means – each and every developer should take the binaries from somewhere and install them in their local GAC. You see the troubles coming in that setup, don’t you?WebGACto the rescue here. It’s essentially GAC shared by WebDAV and enables clients to fetch dependencies from the server, simplifyingdependenciesmanagement for a team. Let’s do our pros/cons math. The benefits:

• GAC is good, standard and proven solution for binaries management. It deals well with versions.
• WebGACis a central binaries repository for a team. Every team member synchronizes with it.
• WebDAV ispopular well-known HTTP extensionwith locking, security management, etc. Working with the Apache WebDAV module is generally straightforward.

Let’s see what won’t work so great with that solution:

• It isn’t a proxy.WebGACcan’t download the dependency you needfroma central repository when you need one. You need to manually download it, add it to WebGAC and only then it becomes available to the team. Not only must you work for every version of every dependency needed, the link to the original file is lost.
• No notion of packages. GAC contains singledlls. You install them one by one. But think about NUnit, as an example. It contains about dozen ofdllsalong with various xml and configuration files. How can you install it toGAC?
• Security is cumbersome. You’ll need to configure Apache Server’s security, and even then it won’t be flexible enough to determine betweendeployer(a user that can publish private dependencies) and promoter (a user that can move dependencies from a private repository to a public one).
• Search is basic. WebDAV by itself only knows about files. It doesn’t care about the structure of thefilename, or about the presence of Strong Names.

看来我们还没有发现我们找什么g for, and then…

Here comes NuGet
This is something else.NuGetdesigned to be “a developer focused package management system for the .NET platform intent on simplifying the process of incorporating third-party libraries into a.NET application during development”. That’s exactly what we need. Let’s look how great it is:

• Manages packages, notdlls.
• ProvidesNuGet Gallery– almost 4.5K (at the time of writing) packages are at your disposal for all your development needs.
• Supports binaryversioning.
• 集成了视觉的圣udio.
• Integrates with your build.
• Integrates with your build server (onlyTeamCityat the moment of writing).

This tool’s like a dream come true. So, what can I mention as downsides? Most of them aredownsidesof the NuGet Gallery, notNuGetitself. The Gallery is a young and relatively small project (just for the sake of comparison, Maven Central is 6 years old and contains more than 290K artifacts) and, as such, it has its downsides:

• The content of submissions to the Gallery is (almost) unverified. Everyone can register, get the API key and start uploading whatever they like. Scary, isn’t it? (yes, very scary).
• Being public, NuGet Gallery can’t be used for inter-teampackagesexchange.Private Remote Feedsare the recommended solution. Next we’ll see if it is good enough.

Working with NuGet Remote Feeds
Remote Feeds, introduced in NuGet 1.4 are crucial need for any development team. It serves a dual purpose: it allowssharing 3rdparty packages that aren’t available onGallery(or even replaces the Gallery for those who can’t trust it) and it serves as a target for internal deployments – both for team collaboration and for other usages, as making packages available to QA, or even serving them to the customers from the outside world (by usingChocolatey, for example). If that’s so right, what’s wrong? Here’s what:

• You saw it coming: It isn’t a proxy. You know the score by now.
• It can’t aggregate. TheNuGetRemote Feedexposingone monolithic repository: the one you have on your machine. It can’t aggregate NuGet packages from remote repositories, or exposenumberof local repositories (separated for security reasons, for example).
• You can’t attach your own metadata. Let’s say you want to annotate some package with compatibility information (e.g.workswith certain browsers). No, can’t do.
• The repository is very simplistic. It doesn’t provide any web interface; it browsable and searchable only from a client – be it Visual Studio or the command line interface (pretty basic by itself).
• Even the VS search interface is very basic (all you have is arbitrary sorting and free text search). It should be enough forstarters butlack of searching inside the packages orbyproperties (from the previous bullet) will bite you eventually.
• The security scheme is even less than simplistic. All that’s required to authenticate a deployment/delete of all the users at once is an API key. What about separation of duties? Some users should only be able to read, others only to annotate with metadata (QA team that tests compatibility in my previous example), and only small subgroup – to deploy. The all-or-nothing scheme is definitely insufficient.
• Storage format is suboptimal:
  • The packages are stored on the filesystem in a naive simple format.Thatfine for small repository,but as you grow, you’d expect storage which more optimized for binaries.
  • The metadata is not indexed. Again, fine forsmall repo, troubles are foreseen when it comes to scaling.

So, is there a good alternative to NuGet Remote Feed to be your in-house Gallery for NuGet packages? We, the proud makers ofArtifactory, believe there is:

Meet Artifactory
Artifactoryis anenterprise-grade Binary Repositorythat centralizes all aspects of managing software binaries. That means that we tackle all the problems mentioned above. Weare developingArtifactory since 2006. Being used by millions of users for storing, sharing and managing binaries, we have gathered great feedback from our users.
That’s what we’ve learned:

• Binary packages are different from sources (by being big and binary) and deserve smart storage.
• Binary packages are usuallyarchives(be it jars, zips,rpmsornupkgs）.他们应该可浏览和搜索the need to download them locally todeveloper’s machine.
• Big public repositories exist on the net, they need to beproxiedsmartly (variations, auditing, managing)
• Users come in different flavors. Their permissions should match possible responsibilities (and in the case of binary packages they are different from other cases).
• A binary repository holds critical information, it should be rock-solid, backed up, and DRP ready.
• Your software ends up being a package. We know how to help you…
  • buildit in a reproducible manner, integrating with your build tools and your build server.
  • stageit to ensure the best quality.
  • distributeit to your customers.

You get the gist behind Artifactory by watchingthis 2.5 minutes YouTube video. If you’re not in the mood for movies (or ran out of popcorn), here’squick recap(click on the image for full size):

As you see, instead of working with a number of NuGet Feeds (NuGet Gallery, Orchard Gallery, Remote Feeds from co-workers and from different teams) developers work with exactly one repository. It simplifies setup and daily work and centralizes management and maintenance.
The work is bi-directional, the users resolve their 3rd party dependencies from Artifactory and deploy their created packaged into it.
Now let’s add a build server to the picture (literally):

Yup, with numbers this time. So, here we go (click on the image for full size):

1. Developers find and fetch new 3rd party packages from Artifactory in Visual Studio. The packages are downloaded from Artifactory to the developer’s machine. If the packages aren’t present in Artifactory it will look for them in remote galleries/feeds. On developermachinespackages.config is updated with the list of used packages.
2. Developers commit their code and packages.config (but not the binaries) to VCS.
3. The Build server (as I already mentioned,TeamCitynow supports NuGet) takes the changes from the VCS.
4. It builds the solution and packs the produced artifacts asNuGetpackages.
5. During the build it fetches the needed packages from Artifactory. If the packages aren’t present in Artifactory it will look for them in remote galleries/feeds.
6. Once the packages are built they are deployed to Artifactory.

Built packages in Artifactory can be used by other teams (as their 3rd party dependencies), byQA forrunning tests and even by the end users (ChocolateyFTW), all this with fine-grained permissions and robust promotion procedures (moving a package between repositories with different visibility rules).
You know what? It deserves dedicated how-to blog post. I’ll link it here once published.

Assuming you’ve read up to this point, you’ve gathered that starting from Artifactory version 2.5.0 we are proud to serve the .NET world with full NuGet support. We can proxy any remote NuGet feed (starting with NuGet Gallery, of course), we can host the packages that aren’t found on any remote NuGet feed, we can host the packages you produce and we can aggregate any number of repositories of any kind under single a URL. We provide you with an awesome UI for configuring your repositories, browsing and searching for your packages. We also feature smart storage that enables attaching searchable metadata on top of your binaries. We can do it allonthe cloud with ourSAAS version.

Hopefully,you’re convinced by now and probably looking for the download link on our site (here’s it, BTW, click on “Evalution”). If not, give it a try by playing withour live demo. Look at thenuget-gallery cache: that’s how we proxy the NuGet Gallery. You’ll find some of the packages saved locally; once you’ve selected a package, you’ll see all kinds of information about it: its name and size, who deployed it to Artifactory, where it came from (from NuGet Gallery, naturally for this is the NuGet Gallery cache) and the operations you can perform on this package (as anonymous the selection is naturally limited). Clicking on the triangle in the tree will open the package and let you dive into its content, including downloading specific files from the archive:

We, at JFrog, believe that Artifactory is the missing piece of the puzzle for a robust, agile .NET dependency management, which can make the development process easier compared to other alternatives.

We’ll be happy to receive any insights, thoughts and comments on the ideas presented in this blog and/or your experience usingArtifactory together withNuGet.