8

I'm working with a company setting up a brand new project, and we're talking about what tools to use for what. I was talking about Artifactory or Nexus for storing built artifacts (APKs in this case), and they asked why they can't just use Bitbucket like they will use for the software, to reduce the number of tools they need to install and maintain.

My initial response was "Source goes in source repo and artifacts go in artifact repo", but that's not actually an answer why or why not. In fact Googling around didn't offer any reasoning either.

ADDING TO THIS: This product is for a regulated industry. That means we need a complete versioned, auditable record of the artifacts built. This is one of the reasons we're looking into this. Also, as I said in some comments but will add here, we will install Bitbucket in a private GCP not accessible by the public, so there are no concerns about others accessing it.

Any input into this? Anything that would bite us later on if we store them in Bitbucket?

9

Reasons not to store large binaries in a git repository:

  • Everbody cloning your repository will download all those binaries, by default. Binaries, if built regularly, tend to consume massive amounts of storage, compared to source code - git cannot compress them, or calculate deltas, to reduce their size.
  • git goes to great lengths to make sure history is not lost, it will never delete anything that is part of the history of any ref (branches or tags). That also means that if you later decide to delete old, long-useless binaries due to running out of space, you will find that, while not impossible, very hard indeed.
  • One point of proper artifact stores is that they help phasing out outdated or compromised parts. git cannot really do that for you - you'd have to write your own tooling for that - and you will never get rid of the old versions in history.
  • The concept of a "commit" does not really apply to binaries. You need operations like "upload" and "download", as they are regularly involved in the build processes of your software (i.e., bundler in the ruby world, or maven in Java). Those builders know how to fetch their 3rd party libraries from artifact repositories easily, and how to upload new versions. They may be convinced to work with a git repository for downloads, but since git has no versioning information, you again need to have your own tooling, specify the commit in which to find that binary manually, or have all binaries ever in use checked out locally. And uploading to a git repository, again, would use your own tooling (creating spurious commits as well).

In total, using git for that is just the wrong tool. If your colleagues are afraid of adding yet another complex tool like Nexus, then at least convince them to use a simple tool instead of git - like some arbitrary (s)ftp/scp repository in a VM somewhere, with https access via a simple webserver.

If you must use git, then at least make sure the build binaries are not committed together with the source code, but in their own part of history. Check out this older answer to see how to create an orphan branch. Those can at least be deleted later, and the binaries themselves removed by garbage collection; and not every client is forced to download all of it all the time.

  • Please see what I added to the original question after reading your question. We need a complete record of all artifacts built. I'm curious about your statement "Git has no versioning information". That's what git does, and versioning the built artifacts is exactly what we need. We're also building APKs not libraries to be included in other builds. WRT checking source and binaries together, yes I would expect to use a different repo for binaries. Thanks. – dj_segfault Oct 6 '18 at 6:57
  • 2
    I meant "versioning information" in the sense of semantics - the artifact repositories know that a single artifact (name) can have multiple versions; give you the "newest" one and so on. From your comment, the fact that you use different repositories is important and should go into the question as well; that would change the whole viewpoint. My answer was assuming that your colleagues want to commit the build binaries along with the source code. – AnoE Oct 6 '18 at 7:57
4

You can use a Git repository (whether it is hosted on Bitbucket or not) as an artifact repository, but you should be aware that:

  • Git was originally made as a version control system for source code, not for (large) binary data, and this is still its main concern. There are extensions that allow working with large files in Git efficiently, e.g. Git LFS, but still: It is not built into the core of Git.
  • A proper artifact repository has features that a Git based service cannot easily provide. For example you probably want to delete older snapshot artifacts, an operation Git is not designed for, but what is a core feature of artifact repositories. Some more of such features are described in the answers to the question What is an artifact repository?.

So while you can use Git/Bitbucket as an artifact repository, it's a bit like driving a screw with a hammer instead of a screwdriver.

0

If you will store apk files inside Bitbucket's repo you need use git or curl it to clone. And if repo private you need have an access to it, size limitation etc.

The Software repository is more comfortable to store binary files. You can create your own mirror of maven, google repos.

  • Interesting angle. As it happens this application is an Android app that will always be side loaded, not from the app store. The repo being private is a plus for us. I agree all of this would be more of an issue if we were talking about libraries to be included in other builds. Thanks for your input. – dj_segfault Oct 5 '18 at 5:22
0

You could use bitbucket pipelines to deploy to the download section and keep your artifacts there. Here is an example from their documentation

  • I found that and thought that was a great idea, but I THINK (haven't found the docs that say definitively) downloads is only for their cloud version. We're installing Bitbucket in our environment. Will that still work? – dj_segfault Oct 5 '18 at 5:10
  • @dj_segfault, Bitbucket Pipelines (and thus probably also the download section) are (currently?) "cloud only", see this question on Atlassians support forum and BSERV-9245 in their bug tracker. – siegi Oct 5 '18 at 6:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.