この記事は Bazel で Go のソースコードをビルドするぞ の延長ですが、 内容がBazel とはあまり関係ない部分なので別記事としました。
上記の記事でも使った以下のリポジトリを使います。
今回の記事では Deployment を使った普通のデプロイについては書かないのでご了承ください。
下準備
デプロイするにあたって必要なツールをセットアップしていきます。
GCP のアカウント作成はここでは説明しません。 新規作成の場合は無料枠が3万円分もらえるはずなのでとりあえず作ってみるとよいでしょう。
下準備が大半を占めてるのではよ本題は入れって人は 飛ばしてください
kubectl Installation
この記事の目的である kustomize を使ってデプロイするだけであれば kubectl は必要ありませんが、 色々確認するために使うのでインストールしておきましょう。
Mac や Windows の場合は DockerDesktop をインストールして Kubernetes を有効にすれば OK です。
Google Cloud SDK の Setup
GKE にデプロイするには Google Cloud SDK をインストールする必要があります。
環境に合わせてセットアップします。
Mac や Linux の場合は少々面倒で アーカイブをダウンロードしてスクリプトを実行する必要があります。
私の場合はホームディレクトリ配下に解凍して google-cloud-sdk/install.sh を実行しました。
google-cloud-sdk/bin/ 配下にコマンドがあるので PATH を設定すれば gcloud コマンドが使えるようになります。
最後に gcloud init を実行すれば OAuth の認証が行われ、GCPのアカウント(プロジェクト) とひも付きます。
クラスタを作成する
Kubernetes で管理する最も大きな単位が クラスタという入れ物です。
クラスタは GUI で作成することもできますが、コマンドでも作成できます。
今回は gcloud コマンドを使って sample-cluster というクラスタを作りました。
$ gcloud container clusters create --num-nodes=2 sample-cluster \ [master] --zone asia-northeast1-a \ --machine-type g1-small \ --min-nodes=2 --max-nodes=3 WARNING: Currently VPC-native is not the default mode during cluster creation. In the future, this will become the default mode and can be disabled using `--no-enable-ip-alias` flag. Use `--[no-]enable-ip-alias` flag to suppress this warning. WARNING: Newly created clusters and node-pools will have node auto-upgrade enabled by default. This can be disabled using the `--no-enable-autoupgrade` flag. WARNING: Starting in 1.12, default node pools in new clusters will have their legacy Compute Engine instance metadata endpoints disabled by default. To create a cluster with legacy instance metadata endpoints disabled in the default node pool, run `clusters create` with the flag `--metadata disable-legacy-endpoints=true`. WARNING: Your Pod address range (`--cluster-ipv4-cidr`) can accommodate at most 1008 node(s). This will enable the autorepair feature for nodes. Please see https://cloud.google.com/kubernetes-engine/docs/node-auto-repair for more information on node autorepairs. Creating cluster sample-cluster in asia-northeast1-a... Cluster is being health-checked (master is healthy)...done. Created [https://container.googleapis.com/v1/projects/righm9/zones/asia-northeast1-a/clusters/sample-cluster]. To inspect the contents of your cluster, go to: https://console.cloud.google.com/kubernetes/workload_/gcloud/asia-northeast1-a/sample-cluster?project=righm9 kubeconfig entry generated for sample-cluster. NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS sample-cluster asia-northeast1-a 1.14.10-gke.27 34.85.37.248 g1-small 1.14.10-gke.27 2 RUNNING
(おそらく)コマンドでの作成が終わると自動的にコンテキストが切り替わります。(少し自信なし)
切り替わっていない場合は kubectl config use-context でスイッチしましょう。
コンテキストとは kubectl が指すクラスタです。
kubectl で確認してみましょう。
$ kubectl config get-contexts [master] CURRENT NAME CLUSTER AUTHINFO NAMESPACE docker-desktop docker-desktop docker-desktop docker-for-desktop docker-desktop docker-desktop * gke_righm9_asia-northeast1-a_sample-cluster gke_righm9_asia-northeast1-a_sample-cluster gke_righm9_asia-northeast1-a_sample-cluster
無事に GKE に作成したクラスタにスイッチされていますね。
この記事では gke_righm9_asia-northeast1-a_sample-cluster
コンテキストに スイッチされている前提で話を進めていきます。
- warning
- ローカルの開発でも Kubernetes を使っている場合はコンテキストを戻すことを忘れないようにしましょう。
kubectl config use-context docker-desktop Switched to context "docker-desktop".
GCR に Push する
GKE で利用するイメージはどこかしらの Dockerレジストリに Push する必要があります。
Bazel で Go のソースコードをビルドするぞ で Docker Hub に Push してあったのですが、なぜかそれを指定しても Pull できなかった(ImagePullBackOff)ので おとなしく GCR に Push することにしました。
docker コマンドで gcr に PUSH するために 認証ヘルパー を使います。
$ gcloud auth configure-docker Adding credentials for all GCR repositories. WARNING: A long list of credential helpers may cause delays running 'docker build'. We recommend passing the registry name to configure only the registry you are using. After update, the following will be written to your Docker config file located at [/Users/righ/.docker/config.json]: { "credHelpers": { "gcr.io": "gcloud", "marketplace.gcr.io": "gcloud", "eu.gcr.io": "gcloud", "us.gcr.io": "gcloud", "staging-k8s.gcr.io": "gcloud", "asia.gcr.io": "gcloud" } } Do you want to continue (Y/n)? Y Docker configuration file updated.
無事認証が終わったら 以下のようにタグを付けて Push するのが一般的な方法です。
$ docker build -t gcr.io/プロジェクトID/イメージ名 ビルドコンテキスト $ docker push gcr.io/プロジェクトID/イメージ名
Bazel を使って GCR にPUSHする
今回使うリポジトリはもともと Bazel を使うためのサンプルなので GCR へ PUSH するための設定もしてあります。これを使って Push してみます。
$ bazel run //gateway:push_to_gcr INFO: Analyzed target //gateway:push_to_gcr (1 packages loaded, 7 targets configured). INFO: Found 1 target... Target //gateway:push_to_gcr up-to-date: bazel-bin/gateway/push_to_gcr.digest bazel-bin/gateway/push_to_gcr INFO: Elapsed time: 0.385s, Critical Path: 0.00s INFO: 0 processes. INFO: Build completed successfully, 2 total actions INFO: Build completed successfully, 2 total actions 2020/05/03 22:46:35 Successfully pushed Docker image to gcr.io/righm9/gateway:latest$ bazel run //echo:push_to_gcr INFO: Analyzed target //echo:push_to_gcr (1 packages loaded, 7 targets configured). INFO: Found 1 target... Target //echo:push_to_gcr up-to-date: bazel-bin/echo/push_to_gcr.digest bazel-bin/echo/push_to_gcr INFO: Elapsed time: 0.318s, Critical Path: 0.00s INFO: 0 processes. INFO: Build completed successfully, 2 total actions INFO: Build completed successfully, 2 total actions 2020/05/03 22:46:55 Successfully pushed Docker image to gcr.io/righm9/echo:latest
こんな感じになっていれば OK です。
kustomize というリポジトリはこの後のセクションで作ります。
kustomize
GKE へのデプロイで kustomize を使うには GoogleCloudPlatform/cloud-builders-community - Github を使います。
Clone して、移動して
$ git clone https://github.com/GoogleCloudPlatform/cloud-builders-community Cloning into 'cloud-builders-community'... remote: Enumerating objects: 3866, done. remote: Total 3866 (delta 0), reused 0 (delta 0), pack-reused 3866 Receiving objects: 100% (3866/3866), 965.20 KiB | 660.00 KiB/s, done. Resolving deltas: 100% (1748/1748), done. $ cd cloud-builders-community/kustomize
以下のように gcloud builds submit でビルドすると kustomize という GCR にリポジトリが作られます。 (順番が前後しますが先程のリポジトリはこれによって作られたものだったのです)
$ gcloud builds submit --config cloudbuild.yaml . Creating temporary tarball archive of 5 file(s) totalling 5.8 KiB before compression. Uploading tarball of [.] to [gs://righm9_cloudbuild/source/1588435618.22-b17a4c75902448f98d85fa42a590cb86.tgz] API [cloudbuild.googleapis.com] not enabled on project [846141039915]. Would you like to enable and retry (this will take a few minutes)? (y/N)? y Enabling service [cloudbuild.googleapis.com] on project [846141039915]... Operation "operations/acf.51554e69-6b85-41df-b101-79b6366136a0" finished successfully. Created [https://cloudbuild.googleapis.com/v1/projects/righm9/builds/42caf256-9873-4886-88b8-0096d239cf71]. Logs are available at [https://console.cloud.google.com/cloud-build/builds/42caf256-9873-4886-88b8-0096d239cf71?project=846141039915]. ------------------------------------------------------------------------------------------------------ REMOTE BUILD OUTPUT ------------------------------------------------------------------------------------------------------ starting build "42caf256-9873-4886-88b8-0096d239cf71" FETCHSOURCE Fetching storage object: gs://righm9_cloudbuild/source/1588435618.22-b17a4c75902448f98d85fa42a590cb86.tgz#1588435620139411 Copying gs://righm9_cloudbuild/source/1588435618.22-b17a4c75902448f98d85fa42a590cb86.tgz#1588435620139411... / [1 files][ 2.3 KiB/ 2.3 KiB] Operation completed over 1 objects/2.3 KiB. BUILD Starting Step #0 Step #0: Already have image (with digest): gcr.io/cloud-builders/docker Step #0: Sending build context to Docker daemon 11.78kB Step #0: Step 1/7 : FROM gcr.io/cloud-builders/gcloud Step #0: ---> f340f9885ea0 Step #0: Step 2/7 : ENV VER 3.4.0 Step #0: ---> Running in d59b849b9bd4 Step #0: Removing intermediate container d59b849b9bd4 Step #0: ---> 5cbc9f7b5b08 Step #0: Step 3/7 : ENV VERSION v${VER} Step #0: ---> Running in f151f29f5ac5 Step #0: Removing intermediate container f151f29f5ac5 Step #0: ---> a0773d5aa34a Step #0: Step 4/7 : COPY kustomize.bash /builder/kustomize.bash Step #0: ---> 63bd42008292 Step #0: Step 5/7 : RUN apt-get update && apt-get install -y wget && wget https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2F${VERSION}/kustomize_${VERSION}_linux_amd64.tar.gz && tar xvzf kustomize_${VERSION}_linux_amd64.tar.gz && mkdir /builder/kustomize && mv kustomize /builder/kustomize/kustomize && chmod +x /builder/kustomize/kustomize && chmod +x /builder/kustomize.bash && rm kustomize_${VERSION}_linux_amd64.tar.gz && apt-get remove --purge -y wget && apt-get --purge -y autoremove && apt-get clean && rm -rf /var/lib/apt/lists/* Step #0: ---> Running in cc446cdec4e6 Step #0: Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [109 kB] Step #0: Get:2 http://ppa.launchpad.net/git-core/ppa/ubuntu xenial InRelease [23.8 kB] Step #0: Get:3 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB] Step #0: Get:4 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [861 kB] Step #0: Get:5 http://security.ubuntu.com/ubuntu xenial-security/main Translation-en [323 kB] Step #0: Get:6 http://security.ubuntu.com/ubuntu xenial-security/restricted amd64 Packages [7204 B] Step #0: Get:7 http://security.ubuntu.com/ubuntu xenial-security/restricted Translation-en [2152 B] Step #0: Get:8 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [490 kB] Step #0: Get:9 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [200 kB] Step #0: Get:10 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [6088 B] Step #0: Get:11 http://security.ubuntu.com/ubuntu xenial-security/multiverse Translation-en [2888 B] Step #0: Get:12 http://ppa.launchpad.net/git-core/ppa/ubuntu xenial/main amd64 Packages [3372 B] Step #0: Get:13 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB] Step #0: Get:14 http://ppa.launchpad.net/git-core/ppa/ubuntu xenial/main Translation-en [2432 B] Step #0: Get:15 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB] Step #0: Get:16 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1201 kB] Step #0: Get:17 http://archive.ubuntu.com/ubuntu xenial/main Translation-en [568 kB] Step #0: Get:18 http://archive.ubuntu.com/ubuntu xenial/restricted amd64 Packages [8344 B] Step #0: Get:19 http://archive.ubuntu.com/ubuntu xenial/restricted Translation-en [2908 B] Step #0: Get:20 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [7532 kB] Step #0: Get:21 http://archive.ubuntu.com/ubuntu xenial/universe Translation-en [4354 kB] Step #0: Get:22 http://archive.ubuntu.com/ubuntu xenial/multiverse amd64 Packages [144 kB] Step #0: Get:23 http://archive.ubuntu.com/ubuntu xenial/multiverse Translation-en [106 kB] Step #0: Get:24 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [1141 kB] Step #0: Get:25 http://archive.ubuntu.com/ubuntu xenial-updates/main Translation-en [432 kB] Step #0: Get:26 http://archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages [7616 B] Step #0: Get:27 http://archive.ubuntu.com/ubuntu xenial-updates/restricted Translation-en [2272 B] Step #0: Get:28 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [797 kB] Step #0: Get:29 http://archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [333 kB] Step #0: Get:30 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [17.1 kB] Step #0: Get:31 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse Translation-en [8632 B] Step #0: Get:32 http://archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [7280 B] Step #0: Get:33 http://archive.ubuntu.com/ubuntu xenial-backports/main Translation-en [4456 B] Step #0: Get:34 http://archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [8064 B] Step #0: Get:35 http://archive.ubuntu.com/ubuntu xenial-backports/universe Translation-en [4328 B] Step #0: Fetched 19.2 MB in 4s (4123 kB/s) Step #0: Reading package lists... Step #0: Reading package lists... Step #0: Building dependency tree... Step #0: Reading state information... Step #0: The following packages were automatically installed and are no longer required: Step #0: cpp cpp-5 gcc-5 libasan2 libatomic1 libc-dev-bin libc6-dev libcc1-0 Step #0: libcilkrts5 libexpat1-dev libgcc-5-dev libgomp1 libisl15 libitm1 liblsan0 Step #0: libmpc3 libmpfr4 libmpx0 libpython-dev libpython2.7 libpython2.7-dev Step #0: libquadmath0 libtsan0 libubsan0 linux-libc-dev manpages-dev Step #0: python-pkg-resources python2.7-dev Step #0: Use 'apt autoremove' to remove them. Step #0: The following NEW packages will be installed: Step #0: wget Step #0: debconf: delaying package configuration, since apt-utils is not installed Step #0: 0 upgraded, 1 newly installed, 0 to remove and 67 not upgraded. Step #0: Need to get 0 B/299 kB of archives. Step #0: After this operation, 905 kB of additional disk space will be used. Step #0: Selecting previously unselected package wget. (Reading database ... 17441 files and directories currently installed.) Step #0: Preparing to unpack .../wget_1.17.1-1ubuntu1.5_amd64.deb ... Step #0: Unpacking wget (1.17.1-1ubuntu1.5) ... Step #0: Setting up wget (1.17.1-1ubuntu1.5) ... Step #0: --2020-05-02 16:08:04-- https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2Fv3.4.0/kustomize_v3.4.0_linux_amd64.tar.gz Step #0: Resolving github.com (github.com)... 140.82.114.3 Step #0: Connecting to github.com (github.com)|140.82.114.3|:443... connected. Step #0: HTTP request sent, awaiting response... 302 Found Step #0: Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/133067498/e3329480-04c6-11ea-8ace-611b2004e838?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200502%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200502T160804Z&X-Amz-Expires=300&X-Amz-Signature=28be438ecd32e7901d3dfe375331ab6072323917f10164bc2c82f3f6114ea49a&X-Amz-SignedHeaders=host&actor_id=0&repo_id=133067498&response-content-disposition=attachment%3B%20filename%3Dkustomize_v3.4.0_linux_amd64.tar.gz&response-content-type=application%2Foctet-stream [following] Step #0: --2020-05-02 16:08:04-- https://github-production-release-asset-2e65be.s3.amazonaws.com/133067498/e3329480-04c6-11ea-8ace-611b2004e838?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200502%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200502T160804Z&X-Amz-Expires=300&X-Amz-Signature=28be438ecd32e7901d3dfe375331ab6072323917f10164bc2c82f3f6114ea49a&X-Amz-SignedHeaders=host&actor_id=0&repo_id=133067498&response-content-disposition=attachment%3B%20filename%3Dkustomize_v3.4.0_linux_amd64.tar.gz&response-content-type=application%2Foctet-stream Step #0: Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.89.148 Step #0: Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.89.148|:443... connected. Step #0: HTTP request sent, awaiting response... 200 OK Step #0: Length: 8496221 (8.1M) [application/octet-stream] Step #0: Saving to: 'kustomize_v3.4.0_linux_amd64.tar.gz' Step #0: Step #0: 0K .......... .......... .......... .......... .......... 0% 870K 9s Step #0: 50K .......... .......... .......... .......... .......... 1% 1.68M 7s Step #0: 100K .......... .......... .......... .......... .......... 1% 99.3M 5s Step #0: 150K .......... .......... .......... .......... .......... 2% 194M 4s Step #0: 200K .......... .......... .......... .......... .......... 3% 1.64M 4s Step #0: 250K .......... .......... .......... .......... .......... 3% 68.2M 3s Step #0: 300K .......... .......... .......... .......... .......... 4% 207M 3s Step #0: 350K .......... .......... .......... .......... .......... 4% 218M 2s Step #0: 400K .......... .......... .......... .......... .......... 5% 1.74M 3s Step #0: 450K .......... .......... .......... .......... .......... 6% 229M 2s Step #0: 500K .......... .......... .......... .......... .......... 6% 219M 2s Step #0: 550K .......... .......... .......... .......... .......... 7% 229M 2s Step #0: 600K .......... .......... .......... .......... .......... 7% 216M 2s Step #0: 650K .......... .......... .......... .......... .......... 8% 231M 2s Step #0: 700K .......... .......... .......... .......... .......... 9% 236M 1s Step #0: 750K .......... .......... .......... .......... .......... 9% 224M 1s Step #0: 800K .......... .......... .......... .......... .......... 10% 221M 1s Step #0: 850K .......... .......... .......... .......... .......... 10% 1.67M 1s Step #0: 900K .......... .......... .......... .......... .......... 11% 184M 1s Step #0: 950K .......... .......... .......... .......... .......... 12% 232M 1s Step #0: 1000K .......... .......... .......... .......... .......... 12% 227M 1s Step #0: 1050K .......... .......... .......... .......... .......... 13% 196M 1s Step #0: 1100K .......... .......... .......... .......... .......... 13% 196M 1s Step #0: 1150K .......... .......... .......... .......... .......... 14% 219M 1s Step #0: 1200K .......... .......... .......... .......... .......... 15% 231M 1s Step #0: 1250K .......... .......... .......... .......... .......... 15% 221M 1s Step #0: 1300K .......... .......... .......... .......... .......... 16% 254M 1s Step #0: 1350K .......... .......... .......... .......... .......... 16% 8.22M 1s Step #0: 1400K .......... .......... .......... .......... .......... 17% 193M 1s Step #0: 1450K .......... .......... .......... .......... .......... 18% 232M 1s Step #0: 1500K .......... .......... .......... .......... .......... 18% 255M 1s Step #0: 1550K .......... .......... .......... .......... .......... 19% 264M 1s Step #0: 1600K .......... .......... .......... .......... .......... 19% 239M 1s Step #0: 1650K .......... .......... .......... .......... .......... 20% 262M 1s Step #0: 1700K .......... .......... .......... .......... .......... 21% 274M 1s Step #0: 1750K .......... .......... .......... .......... .......... 21% 2.68M 1s Step #0: 1800K .......... .......... .......... .......... .......... 22% 75.3M 1s Step #0: 1850K .......... .......... .......... .......... .......... 22% 13.5M 1s Step #0: 1900K .......... .......... .......... .......... .......... 23% 140M 1s Step #0: 1950K .......... .......... .......... .......... .......... 24% 133M 1s Step #0: 2000K .......... .......... .......... .......... .......... 24% 76.4M 1s Step #0: 2050K .......... .......... .......... .......... .......... 25% 255M 1s Step #0: 2100K .......... .......... .......... .......... .......... 25% 274M 1s Step #0: 2150K .......... .......... .......... .......... .......... 26% 263M 1s Step #0: 2200K .......... .......... .......... .......... .......... 27% 235M 1s Step #0: 2250K .......... .......... .......... .......... .......... 27% 268M 1s Step #0: 2300K .......... .......... .......... .......... .......... 28% 284M 1s Step #0: 2350K .......... .......... .......... .......... .......... 28% 265M 1s Step #0: 2400K .......... .......... .......... .......... .......... 29% 41.8M 1s Step #0: 2450K .......... .......... .......... .......... .......... 30% 246M 0s Step #0: 2500K .......... .......... .......... .......... .......... 30% 11.3M 0s Step #0: 2550K .......... .......... .......... .......... .......... 31% 77.5M 0s Step #0: 2600K .......... .......... .......... .......... .......... 31% 225M 0s Step #0: 2650K .......... .......... .......... .......... .......... 32% 284M 0s Step #0: 2700K .......... .......... .......... .......... .......... 33% 273M 0s Step #0: 2750K .......... .......... .......... .......... .......... 33% 270M 0s Step #0: 2800K .......... .......... .......... .......... .......... 34% 257M 0s Step #0: 2850K .......... .......... .......... .......... .......... 34% 268M 0s Step #0: 2900K .......... .......... .......... .......... .......... 35% 287M 0s Step #0: 2950K .......... .......... .......... .......... .......... 36% 283M 0s Step #0: 3000K .......... .......... .......... .......... .......... 36% 221M 0s Step #0: 3050K .......... .......... .......... .......... .......... 37% 266M 0s Step #0: 3100K .......... .......... .......... .......... .......... 37% 281M 0s Step #0: 3150K .......... .......... .......... .......... .......... 38% 283M 0s Step #0: 3200K .......... .......... .......... .......... .......... 39% 254M 0s Step #0: 3250K .......... .......... .......... .......... .......... 39% 270M 0s Step #0: 3300K .......... .......... .......... .......... .......... 40% 271M 0s Step #0: 3350K .......... .......... .......... .......... .......... 40% 278M 0s Step #0: 3400K .......... .......... .......... .......... .......... 41% 233M 0s Step #0: 3450K .......... .......... .......... .......... .......... 42% 10.1M 0s Step #0: 3500K .......... .......... .......... .......... .......... 42% 9.59M 0s Step #0: 3550K .......... .......... .......... .......... .......... 43% 199M 0s Step #0: 3600K .......... .......... .......... .......... .......... 43% 222M 0s Step #0: 3650K .......... .......... .......... .......... .......... 44% 273M 0s Step #0: 3700K .......... .......... .......... .......... .......... 45% 277M 0s Step #0: 3750K .......... .......... .......... .......... .......... 45% 7.97M 0s Step #0: 3800K .......... .......... .......... .......... .......... 46% 7.59M 0s Step #0: 3850K .......... .......... .......... .......... .......... 47% 259M 0s Step #0: 3900K .......... .......... .......... .......... .......... 47% 276M 0s Step #0: 3950K .......... .......... .......... .......... .......... 48% 286M 0s Step #0: 4000K .......... .......... .......... .......... .......... 48% 254M 0s Step #0: 4050K .......... .......... .......... .......... .......... 49% 17.4M 0s Step #0: 4100K .......... .......... .......... .......... .......... 50% 259M 0s Step #0: 4150K .......... .......... .......... .......... .......... 50% 268M 0s Step #0: 4200K .......... .......... .......... .......... .......... 51% 228M 0s Step #0: 4250K .......... .......... .......... .......... .......... 51% 279M 0s Step #0: 4300K .......... .......... .......... .......... .......... 52% 270M 0s Step #0: 4350K .......... .......... .......... .......... .......... 53% 275M 0s Step #0: 4400K .......... .......... .......... .......... .......... 53% 227M 0s Step #0: 4450K .......... .......... .......... .......... .......... 54% 271M 0s Step #0: 4500K .......... .......... .......... .......... .......... 54% 283M 0s Step #0: 4550K .......... .......... .......... .......... .......... 55% 266M 0s Step #0: 4600K .......... .......... .......... .......... .......... 56% 231M 0s Step #0: 4650K .......... .......... .......... .......... .......... 56% 285M 0s Step #0: 4700K .......... .......... .......... .......... .......... 57% 273M 0s Step #0: 4750K .......... .......... .......... .......... .......... 57% 280M 0s Step #0: 4800K .......... .......... .......... .......... .......... 58% 241M 0s Step #0: 4850K .......... .......... .......... .......... .......... 59% 272M 0s Step #0: 4900K .......... .......... .......... .......... .......... 59% 281M 0s Step #0: 4950K .......... .......... .......... .......... .......... 60% 286M 0s Step #0: 5000K .......... .......... .......... .......... .......... 60% 227M 0s Step #0: 5050K .......... .......... .......... .......... .......... 61% 270M 0s Step #0: 5100K .......... .......... .......... .......... .......... 62% 286M 0s Step #0: 5150K .......... .......... .......... .......... .......... 62% 253M 0s Step #0: 5200K .......... .......... .......... .......... .......... 63% 244M 0s Step #0: 5250K .......... .......... .......... .......... .......... 63% 275M 0s Step #0: 5300K .......... .......... .......... .......... .......... 64% 282M 0s Step #0: 5350K .......... .......... .......... .......... .......... 65% 270M 0s Step #0: 5400K .......... .......... .......... .......... .......... 65% 234M 0s Step #0: 5450K .......... .......... .......... .......... .......... 66% 48.6M 0s Step #0: 5500K .......... .......... .......... .......... .......... 66% 14.2M 0s Step #0: 5550K .......... .......... .......... .......... .......... 67% 274M 0s Step #0: 5600K .......... .......... .......... .......... .......... 68% 229M 0s Step #0: 5650K .......... .......... .......... .......... .......... 68% 285M 0s Step #0: 5700K .......... .......... .......... .......... .......... 69% 285M 0s Step #0: 5750K .......... .......... .......... .......... .......... 69% 269M 0s Step #0: 5800K .......... .......... .......... .......... .......... 70% 18.7M 0s Step #0: 5850K .......... .......... .......... .......... .......... 71% 32.6M 0s Step #0: 5900K .......... .......... .......... .......... .......... 71% 278M 0s Step #0: 5950K .......... .......... .......... .......... .......... 72% 285M 0s Step #0: 6000K .......... .......... .......... .......... .......... 72% 258M 0s Step #0: 6050K .......... .......... .......... .......... .......... 73% 285M 0s Step #0: 6100K .......... .......... .......... .......... .......... 74% 253M 0s Step #0: 6150K .......... .......... .......... .......... .......... 74% 281M 0s Step #0: 6200K .......... .......... .......... .......... .......... 75% 173M 0s Step #0: 6250K .......... .......... .......... .......... .......... 75% 128M 0s Step #0: 6300K .......... .......... .......... .......... .......... 76% 141M 0s Step #0: 6350K .......... .......... .......... .......... .......... 77% 267M 0s Step #0: 6400K .......... .......... .......... .......... .......... 77% 244M 0s Step #0: 6450K .......... .......... .......... .......... .......... 78% 287M 0s Step #0: 6500K .......... .......... .......... .......... .......... 78% 10.0M 0s Step #0: 6550K .......... .......... .......... .......... .......... 79% 262M 0s Step #0: 6600K .......... .......... .......... .......... .......... 80% 212M 0s Step #0: 6650K .......... .......... .......... .......... .......... 80% 282M 0s Step #0: 6700K .......... .......... .......... .......... .......... 81% 285M 0s Step #0: 6750K .......... .......... .......... .......... .......... 81% 283M 0s Step #0: 6800K .......... .......... .......... .......... .......... 82% 255M 0s Step #0: 6850K .......... .......... .......... .......... .......... 83% 267M 0s Step #0: 6900K .......... .......... .......... .......... .......... 83% 267M 0s Step #0: 6950K .......... .......... .......... .......... .......... 84% 288M 0s Step #0: 7000K .......... .......... .......... .......... .......... 84% 237M 0s Step #0: 7050K .......... .......... .......... .......... .......... 85% 7.36M 0s Step #0: 7100K .......... .......... .......... .......... .......... 86% 102M 0s Step #0: 7150K .......... .......... .......... .......... .......... 86% 257M 0s Step #0: 7200K .......... .......... .......... .......... .......... 87% 248M 0s Step #0: 7250K .......... .......... .......... .......... .......... 87% 262M 0s Step #0: 7300K .......... .......... .......... .......... .......... 88% 276M 0s Step #0: 7350K .......... .......... .......... .......... .......... 89% 267M 0s Step #0: 7400K .......... .......... .......... .......... .......... 89% 213M 0s Step #0: 7450K .......... .......... .......... .......... .......... 90% 280M 0s Step #0: 7500K .......... .......... .......... .......... .......... 90% 262M 0s Step #0: 7550K .......... .......... .......... .......... .......... 91% 6.19M 0s Step #0: 7600K .......... .......... .......... .......... .......... 92% 11.0M 0s Step #0: 7650K .......... .......... .......... .......... .......... 92% 240M 0s Step #0: 7700K .......... .......... .......... .......... .......... 93% 37.9M 0s Step #0: 7750K .......... .......... .......... .......... .......... 94% 256M 0s Step #0: 7800K .......... .......... .......... .......... .......... 94% 229M 0s Step #0: 7850K .......... .......... .......... .......... .......... 95% 280M 0s Step #0: 7900K .......... .......... .......... .......... .......... 95% 254M 0s Step #0: 7950K .......... .......... .......... .......... .......... 96% 274M 0s Step #0: 8000K .......... .......... .......... .......... .......... 97% 240M 0s Step #0: 8050K .......... .......... .......... .......... .......... 97% 280M 0s Step #0: 8100K .......... .......... .......... .......... .......... 98% 277M 0s Step #0: 8150K .......... .......... .......... .......... .......... 98% 257M 0s Step #0: 8200K .......... .......... .......... .......... .......... 99% 234M 0s Step #0: 8250K .......... .......... .......... .......... ....... 100% 271M=0.3s Step #0: Step #0: 2020-05-02 16:08:05 (27.3 MB/s) - 'kustomize_v3.4.0_linux_amd64.tar.gz' saved [8496221/8496221] Step #0: Step #0: kustomize Step #0: Reading package lists... Step #0: Building dependency tree... Step #0: Reading state information... Step #0: The following packages were automatically installed and are no longer required: Step #0: cpp cpp-5 gcc-5 libasan2 libatomic1 libc-dev-bin libc6-dev libcc1-0 Step #0: libcilkrts5 libexpat1-dev libgcc-5-dev libgomp1 libisl15 libitm1 liblsan0 Step #0: libmpc3 libmpfr4 libmpx0 libpython-dev libpython2.7 libpython2.7-dev Step #0: libquadmath0 libtsan0 libubsan0 linux-libc-dev manpages-dev Step #0: python-pkg-resources python2.7-dev Step #0: Use 'apt autoremove' to remove them. Step #0: The following packages will be REMOVED: Step #0: wget* Step #0: 0 upgraded, 0 newly installed, 1 to remove and 67 not upgraded. Step #0: After this operation, 905 kB disk space will be freed. (Reading database ... 17451 files and directories currently installed.) Step #0: Removing wget (1.17.1-1ubuntu1.5) ... Step #0: Purging configuration files for wget (1.17.1-1ubuntu1.5) ... Step #0: Reading package lists... Step #0: Building dependency tree... Step #0: Reading state information... Step #0: The following packages will be REMOVED: Step #0: cpp* cpp-5* gcc-5* libasan2* libatomic1* libc-dev-bin* libc6-dev* libcc1-0* Step #0: libcilkrts5* libexpat1-dev* libgcc-5-dev* libgomp1* libisl15* libitm1* Step #0: liblsan0* libmpc3* libmpfr4* libmpx0* libpython-dev* libpython2.7* Step #0: libpython2.7-dev* libquadmath0* libtsan0* libubsan0* linux-libc-dev* Step #0: manpages-dev* python-pkg-resources* python2.7-dev* Step #0: 0 upgraded, 0 newly installed, 28 to remove and 63 not upgraded. Step #0: After this operation, 133 MB disk space will be freed. (Reading database ... 17440 files and directories currently installed.) Step #0: Removing cpp (4:5.3.1-1ubuntu1) ... Step #0: Removing gcc-5 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing cpp-5 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing libgcc-5-dev:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing libasan2:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing libatomic1:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing python2.7-dev (2.7.12-1ubuntu0~16.04.9) ... Step #0: Removing libpython-dev:amd64 (2.7.12-1~16.04) ... Step #0: Removing libpython2.7-dev:amd64 (2.7.12-1ubuntu0~16.04.9) ... Step #0: Removing libexpat1-dev:amd64 (2.1.0-7ubuntu0.16.04.5) ... Step #0: Removing libc6-dev:amd64 (2.23-0ubuntu11) ... Step #0: Removing libc-dev-bin (2.23-0ubuntu11) ... Step #0: Removing libcc1-0:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing libcilkrts5:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing libgomp1:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing libisl15:amd64 (0.16.1-1) ... Step #0: Removing libitm1:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing liblsan0:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing libmpc3:amd64 (1.0.3-1) ... Step #0: Purging configuration files for libmpc3:amd64 (1.0.3-1) ... Step #0: Removing libmpfr4:amd64 (3.1.4-1) ... Step #0: Removing libmpx0:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing libpython2.7:amd64 (2.7.12-1ubuntu0~16.04.9) ... Step #0: Removing libquadmath0:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing libtsan0:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing libubsan0:amd64 (5.4.0-6ubuntu1~16.04.12) ... Step #0: Removing linux-libc-dev:amd64 (4.4.0-176.206) ... Step #0: Removing manpages-dev (4.04-2) ... Step #0: Removing python-pkg-resources (20.7.0-1) ... Step #0: Processing triggers for libc-bin (2.23-0ubuntu10) ... Step #0: Removing intermediate container cc446cdec4e6 Step #0: ---> 1324a1aa600e Step #0: Step 6/7 : ENV PATH=/builder/kustomize:$PATH Step #0: ---> Running in b71c623cc1b7 Step #0: Removing intermediate container b71c623cc1b7 Step #0: ---> 63187d0f3177 Step #0: Step 7/7 : ENTRYPOINT ["/builder/kustomize.bash"] Step #0: ---> Running in 8de8de4ea6a1 Step #0: Removing intermediate container 8de8de4ea6a1 Step #0: ---> bd67e188d43b Step #0: Successfully built bd67e188d43b Step #0: Successfully tagged gcr.io/righm9/kustomize:latest Finished Step #0 Starting Step #1 Step #1: Already have image: gcr.io/righm9/kustomize Step #1: {Version:kustomize/v3.4.0 GitCommit:2c9635967a2b1469d605a91a1d040bd27c73ca7d BuildDate:2019-11-12T05:00:57Z GoOs:linux GoArch:amd64} Finished Step #1 PUSH Pushing gcr.io/righm9/kustomize The push refers to repository [gcr.io/righm9/kustomize] 0dfc862af1d8: Preparing fba1ae934d4f: Preparing bf56ea947895: Preparing b40c899868d8: Preparing 84ff92691f90: Preparing 713b651b89c5: Preparing 2bccb5ac4b09: Preparing 713b651b89c5: Waiting 2bccb5ac4b09: Waiting bf56ea947895: Mounted from cloud-builders/git 84ff92691f90: Mounted from cloud-builders/wget b40c899868d8: Mounted from cloud-builders/gsutil 713b651b89c5: Mounted from cloud-builders/wget 2bccb5ac4b09: Mounted from cloud-builders/wget fba1ae934d4f: Pushed 0dfc862af1d8: Pushed latest: digest: sha256:ff21699d3581aa19ce249e695270df2e4c1919c2ee1b9d0851d5dc982cb8bae0 size: 1790 DONE --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ID CREATE_TIME DURATION SOURCE IMAGES STATUS 42caf256-9873-4886-88b8-0096d239cf71 2020-05-02T16:07:46+00:00 32S gs://righm9_cloudbuild/source/1588435618.22-b17a4c75902448f98d85fa42a590cb86.tgz gcr.io/righm9/kustomize (+1 more) SUCCESS
これで kustomize リポジトリができているはずなので確認してみてください。
GKEにDeployする
ようやくここからが本番です。
ここでも gcloud builds submit
コマンドを使ってビルドをしていきます。 先程使った cloudbuild.yaml
と同じファイル名ですが中身は異なり、前のセクションで作成した
kustomize リポジトリを使って production の Kustomization をビルドしています。
$ gcloud builds submit --config cloudbuild.yaml Creating temporary tarball archive of 33 file(s) totalling 28.8 KiB before compression. Some files were not included in the source upload. Check the gcloud log [/Users/righ/.config/gcloud/logs/2020.05.03/22.58.35.984271.log] to see which files and the contents of the default gcloudignore file used (see `$ gcloud topic gcloudignore` to learn more). Uploading tarball of [.] to [gs://righm9_cloudbuild/source/1588514316.06-6d59d2dd01774271bd0131c108cbf44b.tgz] Created [https://cloudbuild.googleapis.com/v1/projects/righm9/builds/b22a262a-6792-427f-aba8-c19b09d1b2a3]. Logs are available at [https://console.cloud.google.com/cloud-build/builds/b22a262a-6792-427f-aba8-c19b09d1b2a3?project=846141039915]. ------------------------------------------------- REMOTE BUILD OUTPUT ------------------------------------------------- starting build "b22a262a-6792-427f-aba8-c19b09d1b2a3" FETCHSOURCE Fetching storage object: gs://righm9_cloudbuild/source/1588514316.06-6d59d2dd01774271bd0131c108cbf44b.tgz#1588514317616235 Error: Exec command aa533ebfef3c9a50f5b5ee4efd886e8c7f2f80b45f8378cc2480eb70dbc9fee3 is already running BUILD Starting Step #0 - "deploy gateway" Step #0 - "deploy gateway": Pulling image: gcr.io/righm9/kustomize Step #0 - "deploy gateway": Using default tag: latest Step #0 - "deploy gateway": latest: Pulling from righm9/kustomize Step #0 - "deploy gateway": 75f546e73d8b: Already exists Step #0 - "deploy gateway": 0f3bb76fc390: Already exists Step #0 - "deploy gateway": 3c2cba919283: Already exists Step #0 - "deploy gateway": 5a992b2091ae: Already exists Step #0 - "deploy gateway": dc685810031f: Already exists Step #0 - "deploy gateway": 0325363c9dcf: Pulling fs layer Step #0 - "deploy gateway": cfe0bb10a3dc: Pulling fs layer Step #0 - "deploy gateway": 0325363c9dcf: Verifying Checksum Step #0 - "deploy gateway": 0325363c9dcf: Download complete Step #0 - "deploy gateway": 0325363c9dcf: Pull complete Step #0 - "deploy gateway": cfe0bb10a3dc: Verifying Checksum Step #0 - "deploy gateway": cfe0bb10a3dc: Download complete Step #0 - "deploy gateway": cfe0bb10a3dc: Pull complete Step #0 - "deploy gateway": Digest: sha256:0a4a22993c7380cc57e145112a22e0cc9ccb2dfb1e1ec376e0ee6d141179b04c Step #0 - "deploy gateway": Status: Downloaded newer image for gcr.io/righm9/kustomize:latest Step #0 - "deploy gateway": gcr.io/righm9/kustomize:latest Step #0 - "deploy gateway": Running: gcloud container clusters get-credentials --project="righm9" --zone="asia-northeast1-a" "sample-cluster" Step #0 - "deploy gateway": Fetching cluster endpoint and auth data. Step #0 - "deploy gateway": kubeconfig entry generated for sample-cluster. Step #0 - "deploy gateway": configmap/nginx-conf unchanged Step #0 - "deploy gateway": service/gateway unchanged Step #0 - "deploy gateway": deployment.apps/gateway created Finished Step #0 - "deploy gateway" Starting Step #1 - "deploy echo" Step #1 - "deploy echo": Already have image (with digest): gcr.io/righm9/kustomize Step #1 - "deploy echo": service/echo unchanged Step #1 - "deploy echo": deployment.apps/echo created Finished Step #1 - "deploy echo" PUSH DONE ----------------------------------------------------------------------------------------------------------------------- ID CREATE_TIME DURATION SOURCE IMAGES STATUS b22a262a-6792-427f-aba8-c19b09d1b2a3 2020-05-03T13:58:39+00:00 18S gs://righm9_cloudbuild/source/1588514316.06-6d59d2dd01774271bd0131c108cbf44b.tgz - SUCCESS
これだけです。
ワークロードを見てみると Push されていることがわかりますね。
production の kustomization.yaml では GCR に Push したイメージを使うようにベースの定義にパッチしています。
最後に gateway の IP にアクセス＆動作確認してみます。
(もうアクセスできません)
無事動いたのでおしまい。
参考
