10

I am building an AWS CodePipeline which depends on two source repositories. If the 'CodeBuild' step is specified with multiple Input artifacts, then you get the error: .. declares 2 input artifacts which is more than the maximum count. Documentation has a strict requirement for CodeBuild to have only a single input artifact.

Lambdas, on another hand, do not have this limitation. I've been looking for a way how a Lambda could "merge" two artifacts into a single one which I can pass to the CodeBuild then. Unfortunately, information on this subject is quite limited.

Does anyone have a working CodePipeline Lambda which would merge two Artifacts into one?

enter image description here

Note: Documentation about How to access artifacts.

4

We've had exactly the same problem - our build stage needs to use information from two separate locations.

To solve this in a generic manner, I've created two general purpose Lambdas - one that will merge artifacts, and one that will extract specific files from an artifact. In both cases a new artifact is created that can be passed on to a subsequent CodePipeline stage.

See https://github.com/tooltwist/codepipeline-artifact-munge

If it does not meet your exact requirement, it should be easy enough to tweak to do what you need.

  • Awesome work, Philip, thanks for sharing this Lambda. – romaninsh Jan 26 '18 at 13:55
4

I have actually solved the problem by cloning repository directly in codebuild:

Pass GitHub token:

      - { Name: GITHUB_TOKEN, Value: {Ref: GitHubToken } }
      - { Name: GITHUB_BRANCH, Value: {Ref: GitHubBranch } }

# Execute git clone in install step:

      - git clone --single-branch --depth=1 -b $GITHUB_BRANCH https://gitorgname:$GITHUB_TOKEN@github.com/gitorgname/reponame.git  src/reponame

I found that this solution is easier to implement, does not rely on Lambdas and even though performs two pulls, works quite reliably.

  • Where exactly do you put the - { Name ... } ? – Alex R Jun 3 '18 at 4:20
  • I create CodeBuild through cloudformation, but that is an environment variable which you can specify through console also. – romaninsh Jun 22 '18 at 14:12
1

As of Sep 4 2018 Amazon added support for multiple input sources:

https://aws.amazon.com/about-aws/whats-new/2018/08/aws-codebuild-adds-ability-to-create-build-projects-with-multiple-input-sources-and-output-artifacts/

From documentation:

{
"name": "sample-project",
"source": {
  "type": "S3",
  "location": "bucket/sample.zip"
},
"secondarySources": [
  {
    "type": "CODECOMMIT",
    "location": "https://git-codecommit.us-west-2.amazonaws.com/v1/repos/repo"
    "sourceIdentifier": "source1"
  },
  {
    "type": "GITHUB",
    "location": "https://github.com/awslabs/aws-codebuild-jenkins-plugin"
    "sourceIdentifier": "source2"
  }
],

version: 0.2

phases:
  build:
    commands:
      - cd $CODEBUILD_SRC_DIR_source1
      - touch file1
      - cd $CODEBUILD_SRC_DIR_source2
      - touch file2

artifacts:
  secondary-artifacts:
    artifact1:
      base-directory: $CODEBUILD_SRC_DIR_source1
      files:
        - file1
    artifact2:
      base-directory: $CODEBUILD_SRC_DIR_source2
      files:
        - file2
-1

You can review CodeBuild. The BuildScpec file can be put in the CodeBuild project.CodeBuild Project

  • 3
    I fail to see how this answer the question. – Tensibai Jun 15 '18 at 8:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.