We are setting up our source and build system in AWS. we will have two persistent VMs hosting source and artifactory. We have a bunch of compilers (a few Gigs) we want available via NFS, is this something that should be hosted in artifactory?
I'm looking for the 'best practice' answer. I can't find much info on the thought behind this so any links would be appreciated.
In many cases compilers are used as-is, already packaged by 3rd parties. When changing the structure of the build systems changing compilers usually translates into switching to a different version, but which is also pre-packaged. If this is your use case the artifact repository offers a single advantage: availability. Even if the provider decides (for whatever reason) to discontinue the version - you still have your copy in the respository.
But if the compilers receive any updates without version changes or if you do any sort of customisation - even if it's just building them without touching any code - you want to store those particular versions in the artifactory to be able to reliably obtain them at any time.
Side note: how exactly you use the compilers can influence the performance of your builds, especially in very large projects with build times which can take hours. You'd have to analyze your builds' structure and/or usage patterns to figure out if it's worth pursuing optimisations and, if so, in which direction(s).
Directly invoking compilers from NFS is convenient - very little configurations required on the build servers. But performance of massive builds can be very sensitive to the NFS performance, of both the NFS server itself and of the network connecting it to the build server(s). Some artifactories may have direct NFS support. You'd have to measure the impact of switching to such service, in some cases it may still be better to use a dedicated NFS server instead. Or maybe switching to VM images that contain the compilers locally installed.
There are a few main reasons why you would use Artifactory (or any other binary repository manager) over a traditional file storage (in your case NFS).
If none of these are applicable in your use-case, I would say that Artifactory would be overkill and the effort to create and maintain it would not out way the value you would receive from it. Although I have used it in the past, and it is a great tool for what it aims to accomplish :)
One solution that might fit your scenario are environment modules. Given that you only need the compilers on Linux, and have access via a network share (nfs/Samba). To load a toolchain into your terminal, just type
module load gcc
and let autocompletion fill in the version number. It is basically Python’s virtualenv for any Linux program/toolchain/SDK. I have used it extensively and can warmly recommend this tool.
Each compiler would have their own folder which is subdivided by version which allows you to have direct access to all versions. Configuration is defined server-side with PATH and other environment variables. User-side, a bash script has to be installed into users’ .bashrc
