Как добавить переменные среды в CodeBuild buildspec.yml?

Question

Как добавить переменные среды в CodeBuild buildspec.yml?

Я пытаюсь использовать шаблон CloudFormation для определения CodeBuild и CodePipeline для автоматизации развертывания статического веб-сайта, размещенного в корзине S3. Чтобы отдать должное, когда это необходимо, я в основном следую шаблону из https://dzone.com/articles/continuous-delivery-to-s3-via-codepipeline-and-cod.

Проблема, которую я не могу решить, заключается в том, что после добавления переменной среды для версии Hugo, которую я хотел бы использовать для создания статических файлов сайта, я получаю сообщение об ошибке из консоли AWS, которое гласит: "Ошибка проверки шаблона: формат шаблона ошибка: неразрешенные зависимости ресурса [HUGO_VERSION] в блоке ресурсов шаблона."

Почему он не принимает переменную среды HUGO_VERSION, которую я определяю в environment_variables? Это версия 0.1 формата, поэтому она немного отличается от текущей версии 0.2, но я прочитал следующую ссылку: https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

Что меня действительно смущает, так это то, что если я удаляю строки с помощью ${HUGO_VERSION}, шаблон принимается очень хорошо - и затем проверка журналов CloudWatch после сборки показывает (из-за команды printenv) HUGO_VERSION=0.49! Что дает?

Первоначально шаблон выглядит следующим образом.

---
AWSTemplateFormatVersion: '2010-09-09'
Description: Pipeline using CodePipeline and CodeBuild for continuous delivery of a single-page application to S3
Parameters:
  SiteBucketName:
    Type: String
    Description: Name of bucket to create to host the website
  GitHubUser:
    Type: String
    Description: GitHub User
    Default: "stelligent"
  GitHubRepo:
    Type: String
    Description: GitHub Repo to pull from. Only the Name. not the URL
    Default: "devops-essentials"
  GitHubBranch:
    Type: String
    Description: GitHub Branch
    Default: "master"
  GitHubToken:
    NoEcho: true
    Type: String
    Description: Secret. It might look something like 9b189a1654643522561f7b3ebd44a1531a4287af OAuthToken with access to Repo. Go to https://github.com/settings/tokens
  BuildType:
    Type: String
    Default: "LINUX_CONTAINER"
    Description: The build container type to use for building the app
  BuildComputeType:
    Type: String
    Default: "BUILD_GENERAL1_SMALL"
    Description: The build compute type to use for building the app
  BuildImage:
    Type: String
    Default: "aws/codebuild/ubuntu-base:14.04"
    Description: The build image to use for building the app
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: "Site Configuration"
        Parameters:
          - SiteBucketName
      - Label:
          default: "GitHub Configuration"
        Parameters:
          - GitHubToken
          - GitHubUser
          - GitHubRepo
          - GitHubBranch
      - Label:
          default: "Build Configuration"
        Parameters:
          - BuildType
          - BuildComputeType
          - BuildImage
    ParameterLabels:
      SiteBucketName:
        default: Name of S3 Bucket to create for website hosting
      GitHubToken:
        default: GitHub OAuth2 Token
      GitHubUser: 
        default: GitHub User/Org Name
      GitHubRepo: 
        default: GitHub Repository Name
      GitHubBranch: 
        default: GitHub Branch Name
      BuildType: 
        default: CodeBuild type
      BuildComputeType: 
        default: CodeBuild instance type
      BuildImage: 
        default: CodeBuild image
Resources:
  CodeBuildRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - codebuild.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: codebuild-service
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action: "*"
            Resource: "*"
          Version: '2012-10-17'
  CodePipelineRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - codepipeline.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: codepipeline-service
        PolicyDocument:
          Statement:
          - Action:
            - codebuild:*
            Resource: "*"
            Effect: Allow
          - Action:
            - s3:GetObject
            - s3:GetObjectVersion
            - s3:GetBucketVersioning
            Resource: "*"
            Effect: Allow
          - Action:
            - s3:PutObject
            Resource:
            - arn:aws:s3:::codepipeline*
            Effect: Allow
          - Action:
            - s3:*
            - cloudformation:*
            - iam:PassRole
            Resource: "*"
            Effect: Allow
          Version: '2012-10-17'
  SiteBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Delete
    Properties:
      AccessControl: PublicRead
      BucketName: !Ref SiteBucketName
      WebsiteConfiguration:
        IndexDocument: index.html
  PipelineBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Delete
  CodeBuildDeploySite:
    Type: AWS::CodeBuild::Project
    DependsOn: CodeBuildRole
    Properties:
      Name: !Sub ${AWS::StackName}-DeploySite
      Description: Deploy site to S3
      ServiceRole: !GetAtt CodeBuildRole.Arn
      Artifacts:
        Type: CODEPIPELINE
      Environment:
        Type: !Ref BuildType
        ComputeType: !Ref BuildComputeType
        Image: !Sub ${BuildImage}
      Source:
        Type: CODEPIPELINE
        BuildSpec: !Sub |
          version: 0.1
          phases:
            post_build:
              commands:
                - aws s3 cp --recursive --acl public-read ./samples s3://${SiteBucketName}/samples 
                - aws s3 cp --recursive --acl public-read ./html s3://${SiteBucketName}/ 
          artifacts:
            type: zip
            files:
              - ./html/index.html
      TimeoutInMinutes: 10
  Pipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      RoleArn: !GetAtt CodePipelineRole.Arn
      Stages:
      - Name: Source
        Actions:
        - InputArtifacts: []
          Name: Source
          ActionTypeId:
            Category: Source
            Owner: ThirdParty
            Version: '1'
            Provider: GitHub
          OutputArtifacts:
          - Name: SourceArtifacts
          Configuration:
            Owner: !Ref GitHubUser
            Repo: !Ref GitHubRepo
            Branch: !Ref GitHubBranch
            OAuthToken: !Ref GitHubToken
          RunOrder: 1
      - Name: Deploy
        Actions:
        - Name: Artifact
          ActionTypeId:
            Category: Build
            Owner: AWS
            Version: '1'
            Provider: CodeBuild
          InputArtifacts:
          - Name: SourceArtifacts
          OutputArtifacts:
          - Name: DeploymentArtifacts
          Configuration:
            ProjectName: !Ref CodeBuildDeploySite
          RunOrder: 1
      ArtifactStore:
        Type: S3
        Location: !Ref PipelineBucket
Outputs:
  PipelineUrl:
    Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline}
    Description: CodePipeline URL
  SiteUrl:
    Value: !GetAtt [SiteBucket, WebsiteURL]
    Description: S3 Website URL

Теперь, когда я пытаюсь добавить переменную окружения для использования Hugo в конвейере, шаблон выглядит следующим образом.

---
AWSTemplateFormatVersion: '2010-09-09'
Description: Pipeline using CodePipeline and CodeBuild for continuous delivery of a single-page application to S3
Parameters:
  SiteBucketName:
    Type: String
    Description: Name of bucket to create to host the website
  GitHubUser:
    Type: String
    Description: GitHub User
    Default: "stelligent"
  GitHubRepo:
    Type: String
    Description: GitHub Repo to pull from. Only the Name. not the URL
    Default: "devops-essentials"
  GitHubBranch:
    Type: String
    Description: GitHub Branch
    Default: "master"
  GitHubToken:
    NoEcho: true
    Type: String
    Description: Secret. It might look something like 9b189a1654643522561f7b3ebd44a1531a4287af OAuthToken with access to Repo. Go to https://github.com/settings/tokens
  BuildType:
    Type: String
    Default: "LINUX_CONTAINER"
    Description: The build container type to use for building the app
  BuildComputeType:
    Type: String
    Default: "BUILD_GENERAL1_SMALL"
    Description: The build compute type to use for building the app
  BuildImage:
    Type: String
    Default: "aws/codebuild/ubuntu-base:14.04"
    Description: The build image to use for building the app
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: "Site Configuration"
        Parameters:
          - SiteBucketName
      - Label:
          default: "GitHub Configuration"
        Parameters:
          - GitHubToken
          - GitHubUser
          - GitHubRepo
          - GitHubBranch
      - Label:
          default: "Build Configuration"
        Parameters:
          - BuildType
          - BuildComputeType
          - BuildImage
    ParameterLabels:
      SiteBucketName:
        default: Name of S3 Bucket to create for website hosting
      GitHubToken:
        default: GitHub OAuth2 Token
      GitHubUser: 
        default: GitHub User/Org Name
      GitHubRepo: 
        default: GitHub Repository Name
      GitHubBranch: 
        default: GitHub Branch Name
      BuildType: 
        default: CodeBuild type
      BuildComputeType: 
        default: CodeBuild instance type
      BuildImage: 
        default: CodeBuild image
Resources:
  CodeBuildRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - codebuild.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: codebuild-service
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action: "*"
            Resource: "*"
          Version: '2012-10-17'
  CodePipelineRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - codepipeline.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: codepipeline-service
        PolicyDocument:
          Statement:
          - Action:
            - codebuild:*
            Resource: "*"
            Effect: Allow
          - Action:
            - s3:GetObject
            - s3:GetObjectVersion
            - s3:GetBucketVersioning
            Resource: "*"
            Effect: Allow
          - Action:
            - s3:PutObject
            Resource:
            - arn:aws:s3:::codepipeline*
            Effect: Allow
          - Action:
            - s3:*
            - cloudformation:*
            - iam:PassRole
            Resource: "*"
            Effect: Allow
          Version: '2012-10-17'
  SiteBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Delete
    Properties:
      AccessControl: PublicRead
      BucketName: !Ref SiteBucketName
      WebsiteConfiguration:
        IndexDocument: index.html
  PipelineBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Delete
  CodeBuildDeploySite:
    Type: AWS::CodeBuild::Project
    DependsOn: CodeBuildRole
    Properties:
      Name: !Sub ${AWS::StackName}-DeploySite
      Description: Deploy site to S3
      ServiceRole: !GetAtt CodeBuildRole.Arn
      Artifacts:
        Type: CODEPIPELINE
      Environment:
        Type: !Ref BuildType
        ComputeType: !Ref BuildComputeType
        Image: !Sub ${BuildImage}
      Source:
        Type: CODEPIPELINE
        BuildSpec: !Sub |
          version: 0.1
          environment_variables:
            plaintext:
              AWS_DEFAULT_REGION: "US-WEST-2"
              HUGO_VERSION: "0.49"
          phases:
            install:
              commands:
                - printenv
                - echo "Install step..."
                - curl -Ls https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_${HUGO_VERSION}_Linux-64bit.tar.gz -o /tmp/hugo.tar.gz
                - tar xf /tmp/hugo.tar.gz -C /tmp
                - mv /tmp/hugo_${HUGO_VERSION}_linux_amd64/hugo_${HUGO_VERSION}_linux_amd64 /usr/bin/hugo
                - rm -rf /tmp/hugo*
            build:
              commands:
                - hugo
            post_build:
              commands:
                - aws s3 cp --recursive --acl public-read ./public s3://${SiteBucketName}
          artifacts:
            type: zip
            files:
              - ./html/index.html
      TimeoutInMinutes: 10
  Pipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      RoleArn: !GetAtt CodePipelineRole.Arn
      Stages:
      - Name: Source
        Actions:
        - InputArtifacts: []
          Name: Source
          ActionTypeId:
            Category: Source
            Owner: ThirdParty
            Version: '1'
            Provider: GitHub
          OutputArtifacts:
          - Name: SourceArtifacts
          Configuration:
            Owner: !Ref GitHubUser
            Repo: !Ref GitHubRepo
            Branch: !Ref GitHubBranch
            OAuthToken: !Ref GitHubToken
          RunOrder: 1
      - Name: Deploy
        Actions:
        - Name: Artifact
          ActionTypeId:
            Category: Build
            Owner: AWS
            Version: '1'
            Provider: CodeBuild
          InputArtifacts:
          - Name: SourceArtifacts
          OutputArtifacts:
          - Name: DeploymentArtifacts
          Configuration:
            ProjectName: !Ref CodeBuildDeploySite
          RunOrder: 1
      ArtifactStore:
        Type: S3
        Location: !Ref PipelineBucket
Outputs:
  PipelineUrl:
    Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline}
    Description: CodePipeline URL
  SiteUrl:
    Value: !GetAtt [SiteBucket, WebsiteURL]
    Description: S3 Website URL

РЕДАКТИРОВАТЬ 10/20

Все еще не решили это. Я пытался следовать советам, данным ниже matsev, но я все еще получаю ту же ошибку проверки. Для полноты, последний шаблон, который я пытаюсь

AWSTemplateFormatVersion: '2010-09-09'
Description: Pipeline using CodePipeline and CodeBuild for continuous delivery of a single-page application to S3
Parameters:
  SiteBucketName:
    Type: String
    Description: Name of bucket to create to host the website
  GitHubUser:
    Type: String
    Description: GitHub User
    Default: "stelligent"
  GitHubRepo:
    Type: String
    Description: GitHub Repo to pull from. Only the Name. not the URL
    Default: "devops-essentials"
  GitHubBranch:
    Type: String
    Description: GitHub Branch
    Default: "master"
  GitHubToken:
    NoEcho: true
    Type: String
    Description: Secret. It might look something like 9b189a1654643522561f7b3ebd44a1531a4287af OAuthToken with access to Repo. Go to https://github.com/settings/tokens
  BuildType:
    Type: String
    Default: "LINUX_CONTAINER"
    Description: The build container type to use for building the app
  BuildComputeType:
    Type: String
    Default: "BUILD_GENERAL1_SMALL"
    Description: The build compute type to use for building the app
  BuildImage:
    Type: String
    Default: "aws/codebuild/ubuntu-base:14.04"
    Description: The build image to use for building the app
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: "Site Configuration"
        Parameters:
          - SiteBucketName
      - Label:
          default: "GitHub Configuration"
        Parameters:
          - GitHubToken
          - GitHubUser
          - GitHubRepo
          - GitHubBranch
      - Label:
          default: "Build Configuration"
        Parameters:
          - BuildType
          - BuildComputeType
          - BuildImage
    ParameterLabels:
      SiteBucketName:
        default: Name of S3 Bucket to create for website hosting
      GitHubToken:
        default: GitHub OAuth2 Token
      GitHubUser: 
        default: GitHub User/Org Name
      GitHubRepo: 
        default: GitHub Repository Name
      GitHubBranch: 
        default: GitHub Branch Name
      BuildType: 
        default: CodeBuild type
      BuildComputeType: 
        default: CodeBuild instance type
      BuildImage: 
        default: CodeBuild image
Resources:
  CodeBuildRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - codebuild.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: codebuild-service
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action: "*"
            Resource: "*"
          Version: '2012-10-17'
  CodePipelineRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - codepipeline.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: codepipeline-service
        PolicyDocument:
          Statement:
          - Action:
            - codebuild:*
            Resource: "*"
            Effect: Allow
          - Action:
            - s3:GetObject
            - s3:GetObjectVersion
            - s3:GetBucketVersioning
            Resource: "*"
            Effect: Allow
          - Action:
            - s3:PutObject
            Resource:
            - arn:aws:s3:::codepipeline*
            Effect: Allow
          - Action:
            - s3:*
            - cloudformation:*
            - iam:PassRole
            Resource: "*"
            Effect: Allow
          Version: '2012-10-17'
  SiteBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Delete
    Properties:
      AccessControl: PublicRead
      BucketName: !Ref SiteBucketName
      WebsiteConfiguration:
        IndexDocument: index.html
  PipelineBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Delete
  CodeBuildDeploySite:
    Type: AWS::CodeBuild::Project
    DependsOn: CodeBuildRole
    Properties:
      Name: !Sub ${AWS::StackName}-DeploySite
      Description: Deploy site to S3
      ServiceRole: !GetAtt CodeBuildRole.Arn
      Artifacts:
        Type: CODEPIPELINE
      Environment:
        Type: !Ref BuildType
        ComputeType: !Ref BuildComputeType
        Image: !Sub ${BuildImage}
        EnvironmentVariables:
          - Name: HUGO_VERSION
            Value: '0.49'
            Type: PLAINTEXT
      Source:
        Type: CODEPIPELINE
        BuildSpec: !Sub |
          version: 0.2
          env:
            variables:
              AWS_DEFAULT_REGION: "US-WEST-2"
          phases:
            install:
              commands:
                - printenv
                - curl -Ls https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_${HUGO_VERSION}_Linux-64bit.tar.gz -o /tmp/hugo.tar.gz
                - tar xf /tmp/hugo.tar.gz -C /tmp
                - mv /tmp/hugo_${HUGO_VERSION}_linux_amd64/hugo_${HUGO_VERSION}_linux_amd64 /usr/bin/hugo
                - rm -rf /tmp/hugo*
            build:
              commands:
                - hugo
            post_build:
              commands:
                - aws s3 cp --recursive --acl public-read ./samples s3://${SiteBucketName}/samples 
                - aws s3 cp --recursive --acl public-read ./html s3://${SiteBucketName}/ 
          artifacts:
            type: zip
            files:
              - ./html/index.html
      TimeoutInMinutes: 10
  Pipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      RoleArn: !GetAtt CodePipelineRole.Arn
      Stages:
      - Name: Source
        Actions:
        - InputArtifacts: []
          Name: Source
          ActionTypeId:
            Category: Source
            Owner: ThirdParty
            Version: '1'
            Provider: GitHub
          OutputArtifacts:
          - Name: SourceArtifacts
          Configuration:
            Owner: !Ref GitHubUser
            Repo: !Ref GitHubRepo
            Branch: !Ref GitHubBranch
            OAuthToken: !Ref GitHubToken
          RunOrder: 1
      - Name: Deploy
        Actions:
        - Name: Artifact
          ActionTypeId:
            Category: Build
            Owner: AWS
            Version: '1'
            Provider: CodeBuild
          InputArtifacts:
          - Name: SourceArtifacts
          OutputArtifacts:
          - Name: DeploymentArtifacts
          Configuration:
            ProjectName: !Ref CodeBuildDeploySite
          RunOrder: 1
      ArtifactStore:
        Type: S3
        Location: !Ref PipelineBucket
Outputs:
  PipelineUrl:
    Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline}
    Description: CodePipeline URL
  SiteUrl:
    Value: !GetAtt [SiteBucket, WebsiteURL]
    Description: S3 Website URL

5

amazon-web-services aws-cloudformation aws-codepipeline aws-codebuild hugo

Источник

user3325401 09 окт '18 в 04:53

2 ответа

Другие вопросы по тегам amazon-web-services aws-cloudformation aws-codepipeline aws-codebuild hugo

user9897266 22 окт '18 в 07:47 2018-10-22 07:47 · Answer 1 · 2018-10-22 07:47

Я считаю, что происходит следующее: CloudFormation пытается разрешить ${HUGO_VERSION} как Parameter шаблона Cloudformation Template, как он есть внутри функции "!Sub".

Из документации AWS по подфункции

Чтобы буквально написать знак доллара и фигурные скобки (${}), добавьте восклицательный знак (!) После открытой фигурной скобки, например ${!Literal}. AWS CloudFormation разрешает этот текст как $ {Literal}.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-sub.html

Поэтому ваша фаза спецификации сборки должна быть похожа на:

      phases:
        install:
          commands:
            - printenv
            - echo "Install step..."
            - curl -Ls https://github.com/gohugoio/hugo/releases/download/v${!HUGO_VERSION}/hugo_${!HUGO_VERSION}_Linux-64bit.tar.gz -o /tmp/hugo.tar.gz
            - tar xf /tmp/hugo.tar.gz -C /tmp
            - mv /tmp/hugo_${!HUGO_VERSION}_linux_amd64/hugo_${!HUGO_VERSION}_linux_amd64 /usr/bin/hugo
            - rm -rf /tmp/hugo*

Надеюсь это поможет!

user303598 09 окт '18 в 07:47 2018-10-09 07:47 · Answer 2 · 2018-10-09 07:47

Пожалуйста, проверьте свойство окружающей среды AWS::CodeBuild::Project в вашем шаблоне CloudFormation. В частности, EnvironmentVariables позволяет указывать переменные среды, например

  CodeBuildDeploySite:
    Type: AWS::CodeBuild::Project
    DependsOn: CodeBuildRole
    Properties:
      Name: !Sub ${AWS::StackName}-DeploySite
      Description: Deploy site to S3
      ServiceRole: !GetAtt CodeBuildRole.Arn
      Artifacts:
        Type: CODEPIPELINE
      Environment:
        Type: !Ref BuildType
        ComputeType: !Ref BuildComputeType
        Image: !Sub ${BuildImage}
        EnvironmentVariables:
          - Name: HUGO_VERSION
            Value: '0.49'
            Type: PLAINTEXT

# More properties...

Теперь вы можете ссылаться на HUGO_VERSION в качестве переменной среды в вашем buildspec.yml файл, например

pre_build:    
  commands:
    - echo HUGO_VERSION $HUGO_VERSION