>>> kops: Building testing/kops 1.26.3-r2 (using abuild 3.11.21-r0) started Sat, 09 Sep 2023 13:45:55 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go bash (1/5) Installing kubernetes (1.28.1-r1) (2/5) Installing go (1.21.1-r0) (3/5) Installing readline (8.2.1-r2) (4/5) Installing bash (5.2.15-r5) Executing bash-5.2.15-r5.post-install (5/5) Installing .makedepends-kops (20230909.134556) Executing busybox-1.36.1-r6.trigger OK: 556 MiB in 115 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Cleaning up tmpdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.26.3.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 73 31.3M 73 23.0M 0 0 23.2M 0 0:00:01 --:--:-- 0:00:01 23.2M 100 31.3M 100 31.3M 0 0 25.6M 0 0:00:01 0:00:01 --:--:-- 25.7M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.26.3.tar.gz >>> kops: Checking sha512sums... kops-1.26.3.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.26.3.tar.gz... go: downloading k8s.io/klog/v2 v2.90.1 go: downloading k8s.io/apimachinery v0.26.2 go: downloading google.golang.org/api v0.111.0 go: downloading k8s.io/cli-runtime v0.26.2 go: downloading github.com/cert-manager/cert-manager v1.11.0 go: downloading k8s.io/client-go v0.26.2 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/gophercloud/gophercloud v1.2.0 go: downloading github.com/aws/aws-sdk-go v1.44.217 go: downloading github.com/prometheus/client_golang v1.14.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading k8s.io/api v0.26.2 go: downloading k8s.io/klog v1.0.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading go.uber.org/multierr v1.10.0 go: downloading github.com/go-logr/logr v1.2.3 go: downloading golang.org/x/oauth2 v0.6.0 go: downloading k8s.io/component-base v0.26.2 go: downloading github.com/spf13/cobra v1.6.1 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.1 go: downloading github.com/spf13/viper v1.15.0 go: downloading sigs.k8s.io/controller-runtime v0.14.5 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/google/go-containerregistry v0.13.0 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading github.com/hetznercloud/hcloud-go v1.41.0 go: downloading golang.org/x/crypto v0.7.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading helm.sh/helm/v3 v3.11.2 go: downloading github.com/pkg/sftp v1.13.5 go: downloading golang.org/x/sync v0.1.0 go: downloading k8s.io/cloud-provider-aws v1.26.1 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading k8s.io/mount-utils v0.26.2 go: downloading k8s.io/utils v0.0.0-20230308161112-d77c459e9343 go: downloading k8s.io/kubectl v0.26.2 go: downloading github.com/google/go-cmp v0.5.9 go: downloading github.com/digitalocean/godo v1.97.0 go: downloading cloud.google.com/go/compute/metadata v0.2.3 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/cloud-provider-gcp/providers v0.26.2 go: downloading github.com/stretchr/testify v1.8.2 go: downloading github.com/prometheus/client_model v0.3.0 go: downloading google.golang.org/protobuf v1.28.1 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading golang.org/x/net v0.8.0 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/google/gofuzz v1.2.0 go: downloading github.com/google/gnostic v0.6.9 go: downloading github.com/prometheus/common v0.39.0 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading cloud.google.com/go/compute v1.18.0 go: downloading github.com/imdario/mergo v0.3.13 go: downloading golang.org/x/term v0.6.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/Azure/azure-sdk-for-go v68.0.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.12 go: downloading k8s.io/apiserver v0.26.2 go: downloading github.com/golang/protobuf v1.5.2 go: downloading k8s.io/kube-openapi v0.0.0-20221207184640-f3cff1453715 go: downloading github.com/Masterminds/sprig/v3 v3.2.3 go: downloading k8s.io/kubelet v0.26.2 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading golang.org/x/time v0.3.0 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/inconshreveable/mousetrap v1.1.0 go: downloading golang.org/x/sys v0.6.0 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading github.com/spf13/afero v1.9.3 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading github.com/scaleway/scaleway-sdk-go v1.0.0-beta.14 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.21 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading golang.org/x/exp v0.0.0-20230307190834-24139beb5833 go: downloading github.com/google/go-tpm-tools v0.3.10 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/spotinst/spotinst-sdk-go v1.334.0 go: downloading github.com/sergi/go-diff v1.3.1 go: downloading google.golang.org/grpc v1.53.0 go: downloading github.com/googleapis/gax-go/v2 v2.7.0 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading k8s.io/gengo v0.0.0-20230306165830-ab3349d207d4 go: downloading github.com/moby/sys/mountinfo v0.6.2 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.2.0 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading golang.org/x/text v0.8.0 go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/gateway-api v0.6.0 go: downloading k8s.io/apiextensions-apiserver v0.26.1 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/kr/fs v0.1.0 go: downloading k8s.io/cloud-provider v0.26.2 go: downloading k8s.io/csi-translation-lib v0.26.0 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.4 go: downloading github.com/subosito/gotenv v1.4.2 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/magiconair/properties v1.8.7 go: downloading github.com/pelletier/go-toml/v2 v2.0.6 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.2.0 go: downloading github.com/huandu/xstrings v1.4.0 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.3.1 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/docker/cli v20.10.21+incompatible go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.1 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading google.golang.org/genproto v0.0.0-20230223222841-637eb2293923 go: downloading go.opencensus.io v0.24.0 go: downloading github.com/go-openapi/swag v0.22.3 go: downloading github.com/google/btree v1.0.1 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/google/go-sev-guest v0.4.1 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/mattn/go-isatty v0.0.17 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/klauspost/compress v1.15.11 go: downloading github.com/opencontainers/image-spec v1.1.0-rc2 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.3 go: downloading github.com/armon/go-metrics v0.4.0 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/miekg/dns v1.1.50 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/mailru/easyjson v0.7.7 go: downloading github.com/go-openapi/jsonreference v0.20.0 go: downloading github.com/emicklei/go-restful/v3 v3.10.1 go: downloading github.com/google/logger v1.1.1 go: downloading github.com/pborman/uuid v1.2.0 go: downloading github.com/moby/term v0.0.0-20221205130635-1aeaba878587 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/docker/docker v20.10.21+incompatible go: downloading github.com/onsi/ginkgo/v2 v2.6.1 go: downloading github.com/onsi/gomega v1.24.2 go: downloading go.uber.org/goleak v1.2.0 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading go.uber.org/zap v1.24.0 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/stretchr/objx v0.5.0 go: downloading github.com/dnaeon/go-vcr v1.2.0 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading github.com/containerd/containerd v1.6.15 go: downloading oras.land/oras-go v1.2.2 go: downloading github.com/distribution/distribution/v3 v3.0.0-20221208165359-362910506bc2 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/docker/docker-credential-helpers v0.7.0 go: downloading github.com/creack/pty v1.1.18 go: downloading github.com/kr/pretty v0.3.1 go: downloading go.uber.org/atomic v1.10.0 go: downloading github.com/docker/go-connections v0.4.0 go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 go: downloading golang.org/x/mod v0.8.0 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/rogpeppe/go-internal v1.9.0 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/felixge/httpsnoop v1.0.3 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/Microsoft/hcsshim v0.9.6 go: downloading cloud.google.com/go/longrunning v0.3.0 go: downloading cloud.google.com/go v0.107.0 go: downloading github.com/containerd/cgroups v1.0.4 go: downloading github.com/Microsoft/go-winio v0.6.0 go: downloading github.com/docker/go-units v0.5.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.26.3/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.26.3/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version=1.26.3 -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0909 13:48:24.939771 51527 addon.go:233] addon test wants to update all nodes I0909 13:48:24.941564 51527 addon.go:233] addon test wants to update workers nodes I0909 13:48:24.941853 51527 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0909 13:48:24.942297 51527 addon.go:266] installing PKI for "test" I0909 13:48:26.020419 51527 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (1.30s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.376s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.080s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] ? k8s.io/kops/pkg/apis/kops/registry [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] ? k8s.io/kops/pkg/configserver [no test files] ? k8s.io/kops/pkg/dump [no test files] ? k8s.io/kops/pkg/kopscontrollerclient [no test files] ? k8s.io/kops/pkg/kubemanifest [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] ? k8s.io/kops/pkg/model/scalewaymodel [no test files] ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] ? k8s.io/kops/pkg/nodeidentity/scaleway [no test files] ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/digitalocean [no test files] ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/resources/scaleway [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/gossip/scaleway [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] ? k8s.io/kops/tests/fuzz [no test files] ? k8s.io/kops/tests/integration/channel/simple [no test files] ? k8s.io/kops/upup/models [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/scaleway [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/scalewaytasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/install [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] ? k8s.io/kops/util/pkg/env [no test files] ? k8s.io/kops/util/pkg/mirrors [no test files] ? k8s.io/kops/util/pkg/tables [no test files] ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestCreateClusterMinimal I0909 13:48:26.577137 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:26.577325 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:26.577368 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:26.577407 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:26.577434 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:26.577484 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:26.577517 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:26.577534 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:26.577564 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:26.577588 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:26.585697 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:26.585759 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:26.622739 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:26.622951 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:26.623353 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:26.635457 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:26.635538 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:26.635565 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:26.635594 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:26.635618 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:26.635650 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:26.635676 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:26.635692 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:26.635716 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:26.635733 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:26.641837 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:26.641890 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:26.750922 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:26.751100 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:26.751393 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:26.756681 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:26.756762 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:26.756791 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:26.756822 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:26.756848 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:26.756882 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:26.756908 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:26.756926 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:26.756951 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:26.756969 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:26.760806 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:26.760889 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:26.808620 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:26.808911 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:26.809385 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:26.820396 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:26.820484 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:26.820528 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:26.820565 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:26.820591 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:26.820625 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:26.820654 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:26.820672 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:26.820695 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:26.820712 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:26.824779 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:26.824822 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:26.940300 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:26.940476 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:26.940783 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:26.946018 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:26.946087 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:26.946113 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:26.946143 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:26.946170 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:26.946201 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:26.946230 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:26.946249 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:26.946272 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:26.946289 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:26.983746 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:26.983800 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.031993 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:27.032301 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:27.032805 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:27.047502 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.047617 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.047657 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.047706 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.047747 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.047800 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.047913 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.047953 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.048030 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.048089 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.054386 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.054461 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.158301 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:27.158579 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:27.159038 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:27.168307 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.168426 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.168472 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.168519 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.168567 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.168641 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.168692 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.168732 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.168780 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.168814 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.180872 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.180967 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.349843 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:27.350014 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:27.350309 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:27.356563 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.356645 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.356674 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.356708 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.356734 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.356767 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.356797 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.356815 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.356841 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.356859 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.360484 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.360549 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.452534 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:27.452675 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:27.453109 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:27.462438 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.462546 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.462587 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.462635 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.462679 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.462731 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.462776 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.462810 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.462851 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.462879 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.467250 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.467313 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.561169 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:27.561342 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:27.561659 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.99s) === RUN TestCreateClusterHetzner I0909 13:48:27.567382 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.567488 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.567519 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.567553 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.567578 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.567611 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.567636 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.567654 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.567676 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.567695 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.571652 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.571709 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0909 13:48:27.612586 52398 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0909 13:48:27.612653 52398 new_cluster.go:1354] Cloud Provider ID: "hetzner" I0909 13:48:27.619558 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.619641 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.619670 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.619700 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.619725 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.619757 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.619783 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.619801 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.619826 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.619843 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.623715 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.623793 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.669721 52398 new_cluster.go:1354] Cloud Provider ID: "hetzner" --- PASS: TestCreateClusterHetzner (0.11s) === RUN TestCreateClusterOpenStack I0909 13:48:27.677931 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.678083 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.678149 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.678220 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.678288 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.678363 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.678434 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.678493 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.678561 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.678615 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.682705 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.682771 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0909 13:48:27.716877 52398 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0909 13:48:27.716945 52398 new_cluster.go:1354] Cloud Provider ID: "openstack" I0909 13:48:27.719497 52398 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 --- PASS: TestCreateClusterOpenStack (0.05s) === RUN TestCreateClusterOpenStackOctavia I0909 13:48:27.731074 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.731204 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.731242 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.731289 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.731328 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.731375 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.731415 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.731442 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.731475 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.731502 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.735476 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.735535 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0909 13:48:27.781829 52398 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0909 13:48:27.781872 52398 new_cluster.go:1354] Cloud Provider ID: "openstack" I0909 13:48:27.783835 52398 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 I0909 13:48:27.783878 52398 subnets.go:199] Assigned CIDR 10.0.0.0/22 to subnet utility-us-test1 --- PASS: TestCreateClusterOpenStackOctavia (0.06s) === RUN TestCreateClusterOpenStackNoDNS I0909 13:48:27.791582 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.791671 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.791700 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.791731 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.791756 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.791791 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.791822 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.791840 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.791863 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.791884 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.822544 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.822602 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0909 13:48:27.869213 52398 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0909 13:48:27.869261 52398 new_cluster.go:1354] Cloud Provider ID: "openstack" I0909 13:48:27.869311 52398 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 I0909 13:48:27.869325 52398 subnets.go:199] Assigned CIDR 10.0.0.0/22 to subnet utility-us-test1 --- PASS: TestCreateClusterOpenStackNoDNS (0.09s) === RUN TestCreateClusterCilium I0909 13:48:27.878104 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.878234 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.878331 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.878632 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.878704 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.878767 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.878815 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.878836 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.878876 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.878923 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.883318 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.883372 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.923967 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:27.924140 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:27.924516 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterCilium (0.05s) === RUN TestCreateClusterOverride I0909 13:48:27.930171 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.930253 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.930279 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.930309 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.930334 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.930364 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.930392 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.930410 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.930434 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.930453 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.934029 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.934075 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.973412 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:27.973977 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:27.974286 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.05s) === RUN TestCreateClusterKubernetesFeatureGates I0909 13:48:27.979488 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:27.979574 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:27.979599 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:27.979627 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:27.979651 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:27.979683 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:27.979711 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:27.979730 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:27.979753 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:27.979771 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:27.983368 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:27.983424 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.039274 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:28.039479 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:28.039836 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterKubernetesFeatureGates (0.07s) === RUN TestCreateClusterComplex I0909 13:48:28.048388 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.048476 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.048503 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.048532 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.048557 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.048591 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.048617 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.048635 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.048657 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.048675 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.052400 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.052441 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.086914 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:28.087086 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:28.087387 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.04s) === RUN TestCreateClusterHA I0909 13:48:28.092435 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.092526 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.092554 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.092582 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.092607 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.092639 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.092666 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.092683 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.092706 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.092723 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.096566 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.096633 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.132540 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:28.132863 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:28.132880 52398 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0909 13:48:28.132891 52398 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0909 13:48:28.133216 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:28.141499 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.141583 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.141611 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.141641 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.141665 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.141696 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.141728 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.141747 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.141774 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.141792 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.145311 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.145355 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.179052 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:28.179363 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:28.179396 52398 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0909 13:48:28.179410 52398 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0909 13:48:28.179732 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.10s) === RUN TestCreateClusterMinimalGCE I0909 13:48:28.190939 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.191033 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.191063 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.191096 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.191123 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.191158 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.191185 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.191204 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.191229 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.191247 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.195177 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.195233 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.237917 52398 new_cluster.go:1354] Cloud Provider ID: "gce" I0909 13:48:28.238025 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 I0909 13:48:28.243628 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.243714 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.243743 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.243775 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.243799 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.243832 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.243874 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.243895 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.243922 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.243940 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.248031 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.248084 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.280165 52398 new_cluster.go:1354] Cloud Provider ID: "gce" I0909 13:48:28.280254 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterMinimalGCE (0.09s) === RUN TestCreateClusterHAGCE I0909 13:48:28.285630 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.285717 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.285742 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.285771 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.285796 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.285829 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.285854 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.285873 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.285898 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.285916 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.289566 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.289626 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.344276 52398 new_cluster.go:1354] Cloud Provider ID: "gce" I0909 13:48:28.344395 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.07s) === RUN TestCreateClusterGCE I0909 13:48:28.355769 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.355861 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.355892 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.355926 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.355952 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.355995 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.356029 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.356047 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.356069 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.356088 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.360179 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.360246 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.453859 52398 new_cluster.go:565] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0909 13:48:28.453938 52398 new_cluster.go:1354] Cloud Provider ID: "gce" I0909 13:48:28.454015 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.10s) === RUN TestCreateClusterHASharedZone I0909 13:48:28.459674 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.459764 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.459794 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.459827 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.459851 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.459884 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.459912 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.459930 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.459955 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.459973 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.463816 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.463888 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0909 13:48:28.548183 52398 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0909 13:48:28.548226 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:28.548475 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:28.548784 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.10s) === RUN TestCreateClusterHASharedZones I0909 13:48:28.555685 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.555777 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.555806 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.555840 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.555865 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.555898 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.555926 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.555944 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.555971 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.555990 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.560013 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.560058 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0909 13:48:28.609917 52398 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0909 13:48:28.609974 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:28.610312 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:28.610329 52398 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0909 13:48:28.610674 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.07s) === RUN TestCreateClusterPrivate I0909 13:48:28.628134 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.628265 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.628306 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.628358 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.628410 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.628464 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.628513 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.628551 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.628593 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.628624 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.633388 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.633457 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.691074 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:28.691309 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:28.691328 52398 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0909 13:48:28.691664 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:28.691699 52398 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.07s) === RUN TestCreateClusterPrivateGCE I0909 13:48:28.698142 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.698235 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.698263 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.698294 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.698319 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.698353 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.698380 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.698397 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.698420 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.698438 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.702352 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.702408 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.841845 52398 new_cluster.go:565] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0909 13:48:28.841930 52398 new_cluster.go:1354] Cloud Provider ID: "gce" I0909 13:48:28.842037 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.15s) === RUN TestCreateClusterWithNGWSpecified I0909 13:48:28.851081 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.851207 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.851243 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.851282 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.851318 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.851361 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.851396 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.851422 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.851453 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.851478 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.855730 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.855773 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.930098 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:28.930321 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:28.930340 52398 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0909 13:48:28.930690 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:28.930724 52398 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.09s) === RUN TestCreateClusterWithINGWSpecified I0909 13:48:28.936851 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.936943 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.936971 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.937005 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.937030 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.937063 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.937092 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.937111 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.937136 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.937154 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.941229 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.941280 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.987425 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:28.987645 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:28.987665 52398 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0909 13:48:28.987985 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:28.988017 52398 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.06s) === RUN TestCreateClusterSharedVPC I0909 13:48:28.994187 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:28.994286 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:28.994316 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:28.994349 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:28.994376 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:28.994410 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:28.994438 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:28.994455 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:28.994479 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:28.994497 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:28.998182 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:28.998239 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.051671 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:29.051830 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.051882 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.051924 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.051942 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.051987 52398 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0909 13:48:29.052279 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.06s) === RUN TestCreateClusterSharedSubnets I0909 13:48:29.057660 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:29.057762 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:29.057791 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:29.057822 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:29.057850 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:29.057884 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:29.057916 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:29.057937 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:29.057962 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:29.057980 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:29.061565 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.061605 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.154308 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.154383 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.154505 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:29.154724 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.154763 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.154802 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.154834 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.155273 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.11s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0909 13:48:29.163448 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:29.163561 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:29.163595 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:29.163643 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:29.163677 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:29.163736 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:29.163773 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:29.163798 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:29.163830 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:29.163897 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:29.169330 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.169423 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.246215 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:48:29.246272 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.246305 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.246407 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:29.246557 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.246585 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.246628 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.246654 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.246976 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.09s) === RUN TestCreateClusterPrivateSharedSubnets I0909 13:48:29.252675 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:29.252771 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:29.252801 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:29.252847 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:29.252882 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:29.252927 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:29.252962 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:29.252994 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:29.253033 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:29.253058 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:29.257190 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.257231 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.316326 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.316403 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.316485 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.316516 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.316613 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:29.316840 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.316876 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.316917 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:29.316942 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:29.317404 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.07s) === RUN TestCreateClusterIPv6 I0909 13:48:29.323384 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:29.323483 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:29.323508 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:29.323535 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:29.323561 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:29.323596 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:29.323624 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:29.323643 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:29.323668 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:29.323686 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:29.328982 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.329046 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.386731 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:29.386938 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet dualstack-us-test-1a I0909 13:48:29.386958 52398 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0909 13:48:29.387536 52398 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.07s) === RUN TestCreateClusterDifferentAMIs I0909 13:48:29.396491 52398 featureflag.go:166] FeatureFlag "APIServerNodes"=true I0909 13:48:29.396512 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:29.396595 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:29.396621 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:29.396651 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:29.396674 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:29.396706 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:29.396732 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:29.396749 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:29.396772 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:29.396788 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:29.402269 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.402307 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.485981 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:29.486069 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:29.486084 52398 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0909 13:48:29.486390 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:29.493377 52398 featureflag.go:166] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.10s) === RUN TestCreateClusterKarpenter I0909 13:48:29.493482 52398 featureflag.go:166] FeatureFlag "Karpenter"=true I0909 13:48:29.493494 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:29.493578 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:29.493605 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:29.493633 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:29.493657 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:29.493690 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:29.493716 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:29.493734 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:29.493758 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:29.493775 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:29.497421 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.497473 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0909 13:48:29.529849 52398 new_cluster.go:1354] Cloud Provider ID: "aws" I0909 13:48:29.530074 52398 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0909 13:48:29.530097 52398 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0909 13:48:29.530119 52398 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0909 13:48:29.530456 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:29.536344 52398 featureflag.go:166] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.04s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I0909 13:48:29.536725 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:29.536806 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:29.536839 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:29.536905 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:29.536941 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:29.536991 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:29.537027 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:29.537061 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:29.537100 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:29.537126 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:29.542296 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I0909 13:48:29.543462 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:29.543547 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:29.543571 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:29.543605 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:29.543638 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:29.543687 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:29.543720 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:29.543756 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:29.543793 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:29.543816 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:29.557677 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:29.565763 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:29.565847 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0909 13:48:31.805794 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:34.080819 52398 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0909 13:48:34.082186 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:34.082236 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:34.082404 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:34.084100 52398 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0909 13:48:34.089042 52398 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0909 13:48:34.090240 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:34.090463 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:34.091876 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:34.092827 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:34.092950 52398 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0909 13:48:34.093447 52398 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0909 13:48:34.093731 52398 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0909 13:48:34.098202 52398 target.go:116] Terraform output is in /tmp/test3916525508/out --- PASS: TestMinimal (4.57s) === RUN TestMinimal_v1_23 I0909 13:48:34.109753 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.109910 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.109965 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.110044 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.110111 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.110202 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.110319 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.110381 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.110535 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.110585 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.136241 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0909 13:48:34.138972 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:34.139061 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:34.957229 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:35.113650 52398 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0909 13:48:35.113970 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:35.114012 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:35.114106 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:35.119848 52398 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0909 13:48:35.122992 52398 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0909 13:48:35.123184 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:35.123497 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:35.124393 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:35.125778 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:35.125884 52398 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0909 13:48:35.126129 52398 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0909 13:48:35.126269 52398 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0909 13:48:35.131297 52398 target.go:116] Terraform output is in /tmp/test3644394923/out --- PASS: TestMinimal_v1_23 (1.03s) === RUN TestMinimal_v1_24 I0909 13:48:35.135006 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:35.135140 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:35.135188 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:35.135241 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:35.135287 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:35.135343 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:35.135430 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:35.135466 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:35.135507 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:35.135537 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:35.155764 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:48:35.157731 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:35.157799 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:35.595160 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:35.713548 52398 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0909 13:48:35.716032 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:35.716069 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:35.716161 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:35.720603 52398 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0909 13:48:35.723623 52398 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0909 13:48:35.723881 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:35.724018 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:35.724788 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:35.726620 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:35.726731 52398 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0909 13:48:35.726967 52398 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0909 13:48:35.727130 52398 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0909 13:48:35.731344 52398 target.go:116] Terraform output is in /tmp/test1553126213/out --- PASS: TestMinimal_v1_24 (0.60s) === RUN TestMinimal_v1_25 I0909 13:48:35.734580 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:35.734712 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:35.734753 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:35.734797 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:35.734838 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:35.734888 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:35.734967 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:35.734996 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:35.735032 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:35.735058 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:35.759425 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0909 13:48:35.762566 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:35.762662 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:36.539640 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:36.667204 52398 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0909 13:48:36.667861 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:36.667901 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:36.667982 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:36.669655 52398 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0909 13:48:36.674894 52398 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0909 13:48:36.675150 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:36.675304 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:36.676697 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:36.677825 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:36.677948 52398 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0909 13:48:36.678212 52398 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0909 13:48:36.678337 52398 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0909 13:48:36.681479 52398 target.go:116] Terraform output is in /tmp/test3528166072/out --- PASS: TestMinimal_v1_25 (0.95s) === RUN TestMinimal_v1_26 I0909 13:48:36.683662 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.683743 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.683769 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.683798 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.683822 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.683878 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.683935 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.683953 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.683977 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.683994 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.705211 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0909 13:48:36.707057 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:36.707124 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:37.445688 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:37.562265 52398 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0909 13:48:37.562882 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:37.562924 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:37.563015 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:37.564525 52398 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0909 13:48:37.569351 52398 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0909 13:48:37.569565 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:37.569806 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:37.571267 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:37.571626 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:37.571694 52398 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0909 13:48:37.571933 52398 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0909 13:48:37.572065 52398 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0909 13:48:37.579165 52398 target.go:116] Terraform output is in /tmp/test4224609258/out --- PASS: TestMinimal_v1_26 (0.90s) === RUN TestMinimal_NoneDNS I0909 13:48:37.581438 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:37.581528 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:37.581571 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:37.581625 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:37.581671 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:37.581722 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:37.581800 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:37.581830 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:37.581865 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:37.581891 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0909 13:48:37.608173 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:37.608259 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:37.803780 52398 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0909 13:48:37.806164 52398 executor.go:111] Tasks: 43 done / 98 total; 20 can run I0909 13:48:37.808003 52398 executor.go:111] Tasks: 63 done / 98 total; 25 can run I0909 13:48:37.808613 52398 executor.go:111] Tasks: 88 done / 98 total; 2 can run I0909 13:48:37.808666 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:37.808738 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:37.808868 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:37.808930 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:37.813234 52398 executor.go:111] Tasks: 90 done / 98 total; 4 can run I0909 13:48:37.813498 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:37.813780 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:37.815273 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:37.829915 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:37.830086 52398 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0909 13:48:37.830408 52398 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0909 13:48:37.830572 52398 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0909 13:48:37.833514 52398 target.go:116] Terraform output is in /tmp/test3122801033/out --- PASS: TestMinimal_NoneDNS (0.25s) === RUN TestHetzner I0909 13:48:37.835659 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:48:38.510614 52398 executor.go:111] Tasks: 0 done / 46 total; 37 can run I0909 13:48:38.512784 52398 executor.go:111] Tasks: 37 done / 46 total; 3 can run I0909 13:48:38.513156 52398 executor.go:111] Tasks: 40 done / 46 total; 2 can run I0909 13:48:38.520884 52398 executor.go:111] Tasks: 42 done / 46 total; 4 can run I0909 13:48:38.522651 52398 executor.go:111] Tasks: 46 done / 46 total; 0 can run I0909 13:48:38.524378 52398 target.go:116] Terraform output is in /tmp/test2247872446/out --- PASS: TestHetzner (0.69s) === RUN TestNvidia I0909 13:48:38.526082 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:38.526178 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:38.526213 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:38.526263 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:38.526303 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:38.526351 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:38.526415 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:38.526445 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:38.526485 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:38.526511 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:38.541055 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:38.542869 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.542940 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.618334 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:38.683242 52398 executor.go:111] Tasks: 0 done / 88 total; 44 can run I0909 13:48:38.684056 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:38.684084 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:38.684164 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:38.689152 52398 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0909 13:48:38.692269 52398 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0909 13:48:38.692456 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.692621 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.693532 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.694822 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.694928 52398 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0909 13:48:38.695145 52398 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0909 13:48:38.695280 52398 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0909 13:48:38.697929 52398 target.go:116] Terraform output is in /tmp/test3057805789/out --- PASS: TestNvidia (0.17s) === RUN TestMinimalGossip I0909 13:48:38.700011 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:38.700090 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:38.700116 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:38.700143 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:38.700167 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:38.700200 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:38.700224 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:38.700241 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:38.700263 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:38.700280 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:38.716041 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.716108 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.859438 52398 executor.go:111] Tasks: 0 done / 86 total; 42 can run I0909 13:48:38.861563 52398 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0909 13:48:38.866017 52398 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0909 13:48:38.866284 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.866488 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.867611 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.867927 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.868002 52398 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0909 13:48:38.868185 52398 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0909 13:48:38.868353 52398 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0909 13:48:38.870989 52398 target.go:116] Terraform output is in /tmp/test3745265629/out --- PASS: TestMinimalGossip (0.17s) === RUN TestMinimalGossipIRSA I0909 13:48:38.872958 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:38.873041 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:38.873068 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:38.873095 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:38.873125 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:38.873157 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:38.873183 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:38.873200 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:38.873222 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:38.873238 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:38.892143 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:38.892204 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:39.028526 52398 executor.go:111] Tasks: 0 done / 89 total; 44 can run I0909 13:48:39.033553 52398 executor.go:111] Tasks: 44 done / 89 total; 20 can run I0909 13:48:39.036530 52398 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0909 13:48:39.036872 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:39.037030 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:39.037628 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:39.038395 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:39.038462 52398 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0909 13:48:39.038637 52398 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0909 13:48:39.038724 52398 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0909 13:48:39.041606 52398 target.go:116] Terraform output is in /tmp/test2427510206/out --- PASS: TestMinimalGossipIRSA (0.17s) === RUN TestMinimalGCE I0909 13:48:39.043722 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:39.044854 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0909 13:48:40.327039 52398 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0909 13:48:40.327198 52398 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0909 13:48:40.327477 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0909 13:48:40.327554 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0909 13:48:40.327903 52398 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0909 13:48:40.327985 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0909 13:48:40.329053 52398 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0909 13:48:40.331376 52398 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0909 13:48:40.334890 52398 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0909 13:48:40.336581 52398 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0909 13:48:40.336753 52398 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0909 13:48:40.338633 52398 target.go:116] Terraform output is in /tmp/test1151032695/out --- PASS: TestMinimalGCE (1.30s) === RUN TestMinimalGCEPrivate I0909 13:48:40.340576 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:40.341662 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0909 13:48:40.582656 52398 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0909 13:48:40.582768 52398 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0909 13:48:40.582928 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0909 13:48:40.582964 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0909 13:48:40.582992 52398 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0909 13:48:40.583075 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0909 13:48:40.584726 52398 executor.go:111] Tasks: 0 done / 65 total; 41 can run I0909 13:48:40.586952 52398 executor.go:111] Tasks: 41 done / 65 total; 17 can run I0909 13:48:40.591712 52398 executor.go:111] Tasks: 58 done / 65 total; 5 can run I0909 13:48:40.596913 52398 executor.go:111] Tasks: 63 done / 65 total; 2 can run I0909 13:48:40.597105 52398 executor.go:111] Tasks: 65 done / 65 total; 0 can run I0909 13:48:40.600411 52398 target.go:116] Terraform output is in /tmp/test2878154065/out --- PASS: TestMinimalGCEPrivate (0.26s) === RUN TestMinimalGCEInternalLoadBalancer I0909 13:48:40.603711 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:40.605887 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0909 13:48:40.871353 52398 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0909 13:48:40.871415 52398 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0909 13:48:40.871502 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0909 13:48:40.871524 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0909 13:48:40.871539 52398 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0909 13:48:40.871586 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0909 13:48:40.871615 52398 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0909 13:48:40.872535 52398 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0909 13:48:40.879362 52398 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0909 13:48:40.882205 52398 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0909 13:48:40.882236 52398 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0909 13:48:40.884256 52398 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0909 13:48:40.884433 52398 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0909 13:48:40.884751 52398 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0909 13:48:40.885014 52398 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0909 13:48:40.887319 52398 target.go:116] Terraform output is in /tmp/test879150762/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.29s) === RUN TestMinimalGCEPublicLoadBalancer I0909 13:48:40.890545 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:40.892483 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0909 13:48:41.176831 52398 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0909 13:48:41.176888 52398 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0909 13:48:41.176966 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com/backups/etcd/main" W0909 13:48:41.176988 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com/backups/etcd/events" W0909 13:48:41.177003 52398 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com" W0909 13:48:41.177055 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0909 13:48:41.178004 52398 executor.go:111] Tasks: 0 done / 70 total; 43 can run I0909 13:48:41.180155 52398 executor.go:111] Tasks: 43 done / 70 total; 18 can run I0909 13:48:41.183287 52398 executor.go:111] Tasks: 61 done / 70 total; 7 can run I0909 13:48:41.185985 52398 executor.go:111] Tasks: 68 done / 70 total; 2 can run I0909 13:48:41.186142 52398 executor.go:111] Tasks: 70 done / 70 total; 0 can run I0909 13:48:41.189087 52398 target.go:116] Terraform output is in /tmp/test4026668154/out --- PASS: TestMinimalGCEPublicLoadBalancer (0.30s) === RUN TestMinimalGCELongClusterName I0909 13:48:41.192016 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:41.194480 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0909 13:48:41.457294 52398 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0909 13:48:41.457500 52398 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0909 13:48:41.457691 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0909 13:48:41.457754 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0909 13:48:41.457807 52398 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0909 13:48:41.457919 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0909 13:48:41.458953 52398 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0909 13:48:41.466566 52398 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0909 13:48:41.469633 52398 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0909 13:48:41.472443 52398 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0909 13:48:41.472632 52398 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0909 13:48:41.474702 52398 target.go:116] Terraform output is in /tmp/test4013994329/out --- PASS: TestMinimalGCELongClusterName (0.28s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0909 13:48:41.476705 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:41.477889 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0909 13:48:41.740085 52398 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0909 13:48:41.740152 52398 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0909 13:48:41.740269 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0909 13:48:41.740292 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0909 13:48:41.740308 52398 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0909 13:48:41.740366 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0909 13:48:41.740403 52398 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0909 13:48:41.741684 52398 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0909 13:48:41.744043 52398 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0909 13:48:41.751092 52398 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0909 13:48:41.751136 52398 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0909 13:48:41.754027 52398 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0909 13:48:41.754203 52398 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0909 13:48:41.754315 52398 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0909 13:48:41.754403 52398 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0909 13:48:41.756408 52398 target.go:116] Terraform output is in /tmp/test2561820290/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.28s) === RUN TestMinimalGCEDNSNone I0909 13:48:41.758357 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:41.759468 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0909 13:48:41.976281 52398 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0909 13:48:41.976367 52398 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0909 13:48:41.976502 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0909 13:48:41.976533 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0909 13:48:41.976556 52398 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0909 13:48:41.976626 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0909 13:48:41.976671 52398 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0909 13:48:41.978219 52398 executor.go:111] Tasks: 0 done / 67 total; 41 can run I0909 13:48:41.980289 52398 executor.go:111] Tasks: 41 done / 67 total; 16 can run W0909 13:48:41.983240 52398 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0909 13:48:41.983286 52398 executor.go:111] Tasks: 57 done / 67 total; 5 can run I0909 13:48:41.988469 52398 executor.go:111] Tasks: 62 done / 67 total; 2 can run I0909 13:48:41.988785 52398 executor.go:111] Tasks: 64 done / 67 total; 1 can run I0909 13:48:41.988987 52398 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0909 13:48:41.989375 52398 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0909 13:48:41.993769 52398 target.go:116] Terraform output is in /tmp/test3367414139/out --- PASS: TestMinimalGCEDNSNone (0.24s) === RUN TestHA I0909 13:48:41.996899 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:41.997029 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:41.997079 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:41.997140 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:41.997203 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:41.997259 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:41.997309 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:41.997345 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:41.997389 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:41.997421 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:42.031366 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:42.034712 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.034796 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.034867 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.034928 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.106771 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:42.202001 52398 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0909 13:48:42.202337 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:42.202374 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:42.202456 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:42.205460 52398 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0909 13:48:42.211532 52398 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0909 13:48:42.211695 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.211826 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.211918 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.212087 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.212597 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.213191 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.213413 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.214198 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.214265 52398 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0909 13:48:42.214562 52398 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0909 13:48:42.214770 52398 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0909 13:48:42.218641 52398 target.go:116] Terraform output is in /tmp/test4175323640/out --- PASS: TestHA (0.22s) === RUN TestHighAvailabilityGCE I0909 13:48:42.221245 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:42.222413 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0909 13:48:42.548427 52398 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0909 13:48:42.548507 52398 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0909 13:48:42.548592 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0909 13:48:42.548614 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0909 13:48:42.548629 52398 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0909 13:48:42.548680 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0909 13:48:42.548730 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0909 13:48:42.548770 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0909 13:48:42.549948 52398 executor.go:111] Tasks: 0 done / 82 total; 49 can run I0909 13:48:42.554611 52398 executor.go:111] Tasks: 49 done / 82 total; 19 can run I0909 13:48:42.560466 52398 executor.go:111] Tasks: 68 done / 82 total; 8 can run I0909 13:48:42.562388 52398 executor.go:111] Tasks: 76 done / 82 total; 6 can run I0909 13:48:42.562617 52398 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0909 13:48:42.566783 52398 target.go:116] Terraform output is in /tmp/test3603781965/out --- PASS: TestHighAvailabilityGCE (0.35s) === RUN TestComplex I0909 13:48:42.570859 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:42.570991 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:42.571039 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:42.571140 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:42.571188 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:42.571247 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:42.571288 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:42.571311 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:42.571357 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:42.571390 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:42.603387 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:48:42.607855 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:42.608012 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.198753 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:43.333202 52398 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0909 13:48:43.333711 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:43.333749 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:43.333853 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:43.338703 52398 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0909 13:48:43.341450 52398 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0909 13:48:43.342003 52398 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0909 13:48:43.342068 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:43.342145 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:43.342251 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:43.342317 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:43.347430 52398 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0909 13:48:43.347617 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.347802 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.348847 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.350756 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.350861 52398 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0909 13:48:43.351110 52398 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0909 13:48:43.351249 52398 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0909 13:48:43.357027 52398 target.go:116] Terraform output is in /tmp/test970121531/out I0909 13:48:43.360569 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:43.360679 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:43.360724 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:43.360770 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:43.360811 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:43.360864 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:43.360908 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:43.360939 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:43.360979 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:43.361008 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:43.396122 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:48:43.400598 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.400669 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.475462 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:43.686274 52398 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0909 13:48:43.686596 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:43.686628 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:43.686715 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:43.688380 52398 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0909 13:48:43.690113 52398 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0909 13:48:43.690598 52398 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0909 13:48:43.690640 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:43.690696 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:43.690788 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:43.690833 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:43.693603 52398 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0909 13:48:43.693712 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.693800 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.694740 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.695325 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.695388 52398 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0909 13:48:43.695563 52398 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0909 13:48:43.695704 52398 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0909 13:48:43.702485 52398 target.go:116] Terraform output is in /tmp/test1433562122/out --- PASS: TestComplex (1.13s) === RUN TestCompress I0909 13:48:43.704736 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:43.704840 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:43.704881 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:43.704933 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:43.704974 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:43.705023 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:43.705065 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:43.705095 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:43.705138 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:43.705164 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:43.720004 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:43.721858 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.721933 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.793238 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:43.868867 52398 executor.go:111] Tasks: 0 done / 86 total; 42 can run I0909 13:48:43.869101 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:43.869128 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:43.869194 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:43.872593 52398 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0909 13:48:43.878490 52398 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0909 13:48:43.878773 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.878855 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.881028 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.881603 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.881673 52398 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0909 13:48:43.881838 52398 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0909 13:48:43.882010 52398 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0909 13:48:43.885826 52398 target.go:116] Terraform output is in /tmp/test1790471100/out --- PASS: TestCompress (0.18s) === RUN TestExternalPolicies I0909 13:48:43.888800 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:43.888915 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:43.888953 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:43.889004 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:43.889046 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:43.889093 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:43.889134 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:43.889163 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:43.889200 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:43.889227 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:43.915781 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:43.919029 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:43.919118 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.031528 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:44.110389 52398 executor.go:111] Tasks: 0 done / 100 total; 45 can run I0909 13:48:44.110658 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:44.110680 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:44.110766 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:44.112607 52398 executor.go:111] Tasks: 45 done / 100 total; 18 can run I0909 13:48:44.114486 52398 executor.go:111] Tasks: 63 done / 100 total; 26 can run I0909 13:48:44.114959 52398 executor.go:111] Tasks: 89 done / 100 total; 3 can run W0909 13:48:44.115017 52398 api.go:53] PageSize not implemented W0909 13:48:44.115055 52398 api.go:53] PageSize not implemented I0909 13:48:44.117698 52398 executor.go:111] Tasks: 92 done / 100 total; 4 can run I0909 13:48:44.117819 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.117972 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.118807 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.119233 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.119317 52398 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0909 13:48:44.119494 52398 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0909 13:48:44.119642 52398 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0909 13:48:44.122700 52398 target.go:116] Terraform output is in /tmp/test3342068683/out --- PASS: TestExternalPolicies (0.24s) === RUN TestMinimalIPv6 I0909 13:48:44.126928 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:44.127014 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:44.127043 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:44.127075 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:44.127105 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:44.127139 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:44.127211 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:44.127231 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:44.127255 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:44.127272 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:44.148173 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:48:44.150301 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.150365 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.230303 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:44.375296 52398 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0909 13:48:44.375739 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:44.375771 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:44.375860 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:44.377553 52398 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0909 13:48:44.379375 52398 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0909 13:48:44.379827 52398 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0909 13:48:44.380138 52398 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0909 13:48:44.380244 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:44.380321 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:44.380468 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:44.380533 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:44.385453 52398 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0909 13:48:44.385650 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.385849 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.387222 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.387480 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.387554 52398 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0909 13:48:44.387767 52398 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0909 13:48:44.387887 52398 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0909 13:48:44.393267 52398 target.go:116] Terraform output is in /tmp/test2596971965/out --- PASS: TestMinimalIPv6 (0.27s) === RUN TestMinimalIPv6Calico I0909 13:48:44.398160 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:44.398292 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:44.398337 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:44.398415 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:44.398458 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:44.398524 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:44.398614 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:44.398649 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:44.398696 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:44.398722 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:44.427523 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:48:44.430774 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.430860 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.502206 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:44.835695 52398 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0909 13:48:44.837186 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:44.837233 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:44.837324 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:44.842112 52398 executor.go:111] Tasks: 47 done / 126 total; 21 can run I0909 13:48:44.843942 52398 executor.go:111] Tasks: 68 done / 126 total; 33 can run I0909 13:48:44.844467 52398 executor.go:111] Tasks: 101 done / 126 total; 9 can run I0909 13:48:44.844723 52398 executor.go:111] Tasks: 110 done / 126 total; 8 can run I0909 13:48:44.844763 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:44.844819 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:44.844951 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:44.844997 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:44.847750 52398 executor.go:111] Tasks: 118 done / 126 total; 4 can run I0909 13:48:44.847858 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.847999 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.848800 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.849568 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.849635 52398 executor.go:111] Tasks: 122 done / 126 total; 2 can run I0909 13:48:44.849815 52398 executor.go:111] Tasks: 124 done / 126 total; 2 can run I0909 13:48:44.849940 52398 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0909 13:48:44.853617 52398 target.go:116] Terraform output is in /tmp/test2125774872/out --- PASS: TestMinimalIPv6Calico (0.46s) === RUN TestMinimalIPv6Cilium I0909 13:48:44.859146 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:44.859278 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:44.859313 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:44.859349 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:44.859381 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:44.859420 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:44.859484 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:44.859506 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:44.859533 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:44.859554 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:44.880718 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:48:44.883340 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.883431 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:44.954495 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:45.090207 52398 executor.go:111] Tasks: 0 done / 125 total; 47 can run I0909 13:48:45.091001 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:45.091039 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:45.091120 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:45.092360 52398 executor.go:111] Tasks: 47 done / 125 total; 21 can run I0909 13:48:45.097274 52398 executor.go:111] Tasks: 68 done / 125 total; 32 can run I0909 13:48:45.097724 52398 executor.go:111] Tasks: 100 done / 125 total; 9 can run I0909 13:48:45.097986 52398 executor.go:111] Tasks: 109 done / 125 total; 8 can run I0909 13:48:45.098039 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:45.098106 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:45.098126 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:45.098182 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:45.100911 52398 executor.go:111] Tasks: 117 done / 125 total; 4 can run I0909 13:48:45.101059 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.101236 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.102433 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.103788 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.103898 52398 executor.go:111] Tasks: 121 done / 125 total; 2 can run I0909 13:48:45.104117 52398 executor.go:111] Tasks: 123 done / 125 total; 2 can run I0909 13:48:45.104272 52398 executor.go:111] Tasks: 125 done / 125 total; 0 can run I0909 13:48:45.109498 52398 target.go:116] Terraform output is in /tmp/test4026133436/out --- PASS: TestMinimalIPv6Cilium (0.25s) === RUN TestMinimalIPv6NoSubnetPrefix I0909 13:48:45.112900 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:45.113001 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:45.113035 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:45.113080 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:45.113117 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:45.113161 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:45.113234 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:45.113263 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:45.113299 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:45.113323 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:45.140309 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:48:45.143825 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.144066 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.216115 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:45.348203 52398 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0909 13:48:45.348847 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:45.348887 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:45.348955 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:45.351656 52398 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0909 13:48:45.355093 52398 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0909 13:48:45.355706 52398 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0909 13:48:45.355959 52398 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0909 13:48:45.356022 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:45.356075 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:45.356106 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:45.356143 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:45.358744 52398 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0909 13:48:45.358855 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.358953 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.359901 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.360723 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.360804 52398 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0909 13:48:45.360977 52398 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0909 13:48:45.361128 52398 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0909 13:48:45.364574 52398 target.go:116] Terraform output is in /tmp/test1894123421/out --- PASS: TestMinimalIPv6NoSubnetPrefix (0.25s) === RUN TestMinimalWarmPool I0909 13:48:45.366869 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:45.366947 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:45.366973 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:45.367005 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:45.367029 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:45.367061 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:45.367113 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:45.367132 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:45.367156 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:45.367184 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:45.381680 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:45.383616 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.383672 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.453891 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:45.575564 52398 executor.go:111] Tasks: 0 done / 89 total; 45 can run I0909 13:48:45.576221 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:45.576249 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:45.576313 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:45.577789 52398 executor.go:111] Tasks: 45 done / 89 total; 19 can run I0909 13:48:45.583736 52398 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0909 13:48:45.583981 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.584130 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.584821 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.587158 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.587271 52398 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0909 13:48:45.587482 52398 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0909 13:48:45.587609 52398 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0909 13:48:45.592019 52398 target.go:116] Terraform output is in /tmp/test221759701/out --- PASS: TestMinimalWarmPool (0.23s) === RUN TestMinimalEtcd I0909 13:48:45.595381 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:45.595490 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:45.595527 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:45.595568 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:45.595607 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:45.595657 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:45.595700 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:45.595731 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:45.595772 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:45.595800 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:45.622917 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:45.625866 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.625942 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.696363 52398 zones.go:93] ListHostedZonesPages { } W0909 13:48:45.754539 52398 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0909 13:48:45.754594 52398 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 10 W0909 13:48:45.756214 52398 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc I0909 13:48:45.767471 52398 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0909 13:48:45.768085 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:45.768113 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:45.768196 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:45.769531 52398 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0909 13:48:45.772417 52398 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0909 13:48:45.772659 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.772768 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.774372 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.775092 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.775176 52398 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0909 13:48:45.775397 52398 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0909 13:48:45.775512 52398 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0909 13:48:45.778185 52398 target.go:116] Terraform output is in /tmp/test2262468934/out --- PASS: TestMinimalEtcd (0.18s) === RUN TestMinimalGp3 I0909 13:48:45.780234 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:45.780325 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:45.780355 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:45.780392 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:45.780420 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:45.780453 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:45.780480 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:45.780500 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:45.780535 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:45.780553 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:45.799017 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:45.800883 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.800944 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.876411 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:45.938317 52398 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0909 13:48:45.938929 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:45.938952 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:45.939009 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:45.940314 52398 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0909 13:48:45.943202 52398 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0909 13:48:45.943953 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.944351 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.947426 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.948446 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.948556 52398 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0909 13:48:45.948794 52398 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0909 13:48:45.948948 52398 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0909 13:48:45.951611 52398 target.go:116] Terraform output is in /tmp/test3127169780/out --- PASS: TestMinimalGp3 (0.17s) === RUN TestMinimalLongClusterName I0909 13:48:45.953787 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:45.953881 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:45.953908 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:45.953936 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:45.953960 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:45.953994 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:45.954021 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:45.954039 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:45.954062 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:45.954079 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:45.968394 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:45.970098 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:45.970156 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0909 13:48:46.040855 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:46.103509 52398 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0909 13:48:46.103894 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:46.103916 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:46.103978 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:46.107209 52398 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0909 13:48:46.111398 52398 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0909 13:48:46.111623 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.111809 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.112625 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.113729 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.113825 52398 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0909 13:48:46.114046 52398 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0909 13:48:46.114188 52398 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0909 13:48:46.116907 52398 target.go:116] Terraform output is in /tmp/test2504731362/out --- PASS: TestMinimalLongClusterName (0.17s) === RUN TestExistingSG I0909 13:48:46.118983 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:46.119060 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:46.119086 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:46.119115 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:46.119148 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:46.119180 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:46.119204 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:46.119220 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:46.119243 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:46.119262 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:46.138379 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:46.140359 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.140419 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.140454 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.140488 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.211465 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:46.281069 52398 executor.go:111] Tasks: 0 done / 146 total; 53 can run I0909 13:48:46.281416 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:46.281451 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:46.281520 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:46.282952 52398 executor.go:111] Tasks: 53 done / 146 total; 22 can run I0909 13:48:46.284751 52398 executor.go:111] Tasks: 75 done / 146 total; 50 can run I0909 13:48:46.285298 52398 executor.go:111] Tasks: 125 done / 146 total; 5 can run W0909 13:48:46.285346 52398 api.go:53] PageSize not implemented W0909 13:48:46.285396 52398 api.go:53] PageSize not implemented W0909 13:48:46.285458 52398 api.go:53] PageSize not implemented W0909 13:48:46.285575 52398 api.go:53] PageSize not implemented I0909 13:48:46.288619 52398 executor.go:111] Tasks: 130 done / 146 total; 8 can run I0909 13:48:46.288746 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.288865 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.289039 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.289230 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.290989 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.293483 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.293867 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.294124 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.294212 52398 executor.go:111] Tasks: 138 done / 146 total; 4 can run I0909 13:48:46.294538 52398 executor.go:111] Tasks: 142 done / 146 total; 4 can run I0909 13:48:46.294727 52398 executor.go:111] Tasks: 146 done / 146 total; 0 can run I0909 13:48:46.301300 52398 target.go:116] Terraform output is in /tmp/test436501755/out --- PASS: TestExistingSG (0.19s) === RUN TestBastionAdditionalUserData I0909 13:48:46.305376 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:46.305489 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:46.305527 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:46.305574 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:46.305615 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:46.305661 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:46.305699 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:46.305727 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:46.305765 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:46.305790 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:46.332060 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:46.334837 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.334911 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.334970 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.407408 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:46.474148 52398 executor.go:111] Tasks: 0 done / 121 total; 46 can run I0909 13:48:46.474807 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:46.474845 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:46.474913 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:46.476433 52398 executor.go:111] Tasks: 46 done / 121 total; 26 can run I0909 13:48:46.478470 52398 executor.go:111] Tasks: 72 done / 121 total; 31 can run I0909 13:48:46.478928 52398 executor.go:111] Tasks: 103 done / 121 total; 5 can run W0909 13:48:46.478972 52398 api.go:53] PageSize not implemented W0909 13:48:46.478999 52398 api.go:53] PageSize not implemented W0909 13:48:46.479131 52398 api.go:53] PageSize not implemented I0909 13:48:46.481713 52398 executor.go:111] Tasks: 108 done / 121 total; 7 can run I0909 13:48:46.481847 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.481954 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.482032 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.482848 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.483027 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.483756 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.483831 52398 executor.go:111] Tasks: 115 done / 121 total; 3 can run I0909 13:48:46.484069 52398 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0909 13:48:46.484249 52398 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0909 13:48:46.491165 52398 target.go:116] Terraform output is in /tmp/test2982610067/out --- PASS: TestBastionAdditionalUserData (0.19s) === RUN TestPrivateWeave I0909 13:48:46.493560 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:46.493649 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:46.493680 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:46.493712 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:46.493739 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:46.493774 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:46.493829 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:46.493863 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:46.493888 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:46.493906 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:46.510450 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:46.512253 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.512310 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.512347 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.592969 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:46.676222 52398 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0909 13:48:46.676734 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:46.676769 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:46.676861 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:46.678410 52398 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0909 13:48:46.680276 52398 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0909 13:48:46.680633 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.680723 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.680782 52398 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0909 13:48:46.680836 52398 api.go:53] PageSize not implemented W0909 13:48:46.680871 52398 api.go:53] PageSize not implemented I0909 13:48:46.683476 52398 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0909 13:48:46.683564 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.683665 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.684587 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.684889 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.684956 52398 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0909 13:48:46.685140 52398 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0909 13:48:46.685256 52398 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0909 13:48:46.688652 52398 target.go:116] Terraform output is in /tmp/test1900983498/out --- PASS: TestPrivateWeave (0.20s) === RUN TestPrivateFlannel I0909 13:48:46.690781 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:46.690866 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:46.690891 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:46.690920 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:46.690945 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:46.690977 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:46.691026 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:46.691044 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:46.691068 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:46.691086 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:46.709705 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:48:46.711805 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.711862 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.711896 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.782914 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:46.932369 52398 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0909 13:48:46.932631 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:46.932658 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:46.932735 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:46.934124 52398 executor.go:111] Tasks: 50 done / 123 total; 26 can run I0909 13:48:46.936041 52398 executor.go:111] Tasks: 76 done / 123 total; 32 can run I0909 13:48:46.936198 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.936330 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.936514 52398 executor.go:111] Tasks: 108 done / 123 total; 5 can run W0909 13:48:46.936625 52398 api.go:53] PageSize not implemented W0909 13:48:46.936648 52398 api.go:53] PageSize not implemented I0909 13:48:46.941028 52398 executor.go:111] Tasks: 113 done / 123 total; 6 can run I0909 13:48:46.941326 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.941571 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.943384 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.944527 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.944623 52398 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0909 13:48:46.944832 52398 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0909 13:48:46.944982 52398 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0909 13:48:46.948482 52398 target.go:116] Terraform output is in /tmp/test3795893110/out --- PASS: TestPrivateFlannel (0.26s) === RUN TestPrivateCalico I0909 13:48:46.951191 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:46.951285 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:46.951318 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:46.951380 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:46.951408 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:46.951442 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:46.951473 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:46.951491 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:46.951517 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:46.951535 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:46.968651 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:48:46.970705 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.970768 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:46.970808 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.051254 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:47.402400 52398 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0909 13:48:47.402957 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:47.402983 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:47.403047 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:47.404577 52398 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0909 13:48:47.407028 52398 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0909 13:48:47.407391 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.407514 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.407579 52398 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0909 13:48:47.407650 52398 api.go:53] PageSize not implemented W0909 13:48:47.407747 52398 api.go:53] PageSize not implemented I0909 13:48:47.410444 52398 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0909 13:48:47.410612 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.410839 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.413876 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.415634 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.415709 52398 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0909 13:48:47.415881 52398 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0909 13:48:47.416030 52398 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0909 13:48:47.419831 52398 target.go:116] Terraform output is in /tmp/test1090790698/out --- PASS: TestPrivateCalico (0.47s) === RUN TestPrivateCilium I0909 13:48:47.422753 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:47.422830 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:47.422855 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:47.422881 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:47.422904 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:47.422936 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:47.422986 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:47.423003 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:47.423025 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:47.423043 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:47.438947 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:47.440871 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.440923 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.440959 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.526070 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:47.633543 52398 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0909 13:48:47.634268 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:47.634298 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:47.634387 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:47.635775 52398 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0909 13:48:47.637735 52398 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0909 13:48:47.638034 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.638157 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.638247 52398 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0909 13:48:47.638357 52398 api.go:53] PageSize not implemented W0909 13:48:47.638457 52398 api.go:53] PageSize not implemented I0909 13:48:47.641033 52398 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0909 13:48:47.641132 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.641204 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.641997 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.642440 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.642507 52398 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0909 13:48:47.642676 52398 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0909 13:48:47.642785 52398 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0909 13:48:47.646304 52398 target.go:116] Terraform output is in /tmp/test1176069656/out --- PASS: TestPrivateCilium (0.23s) === RUN TestPrivateCilium2 I0909 13:48:47.652143 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:47.652270 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:47.652309 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:47.652356 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:47.652402 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:47.652451 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:47.652533 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:47.652564 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:47.652604 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:47.652632 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:47.682884 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:48:47.686366 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.686451 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.686511 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:47.757344 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:48.227622 52398 executor.go:111] Tasks: 0 done / 124 total; 51 can run I0909 13:48:48.228243 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:48.228272 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:48.228370 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:48.234214 52398 executor.go:111] Tasks: 51 done / 124 total; 26 can run I0909 13:48:48.237253 52398 executor.go:111] Tasks: 77 done / 124 total; 32 can run I0909 13:48:48.237670 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.237772 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.237833 52398 executor.go:111] Tasks: 109 done / 124 total; 5 can run W0909 13:48:48.237860 52398 api.go:53] PageSize not implemented W0909 13:48:48.237908 52398 api.go:53] PageSize not implemented I0909 13:48:48.242739 52398 executor.go:111] Tasks: 114 done / 124 total; 6 can run I0909 13:48:48.242905 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.243090 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.244401 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.244557 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.244624 52398 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0909 13:48:48.244834 52398 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0909 13:48:48.244958 52398 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0909 13:48:48.251396 52398 target.go:116] Terraform output is in /tmp/test1576923044/out --- PASS: TestPrivateCilium2 (0.60s) === RUN TestPrivateCiliumAdvanced I0909 13:48:48.256671 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:48.256782 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:48.256816 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:48.256860 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:48.256900 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:48.256952 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:48.257029 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:48.257060 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:48.257113 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:48.257142 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:48.284198 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:48.286410 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.286469 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.286505 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.356666 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:48.461315 52398 executor.go:111] Tasks: 0 done / 126 total; 53 can run I0909 13:48:48.461684 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:48.461708 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:48.461768 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:48.466096 52398 executor.go:111] Tasks: 53 done / 126 total; 26 can run I0909 13:48:48.468643 52398 executor.go:111] Tasks: 79 done / 126 total; 32 can run I0909 13:48:48.469010 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.469122 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.469189 52398 executor.go:111] Tasks: 111 done / 126 total; 5 can run W0909 13:48:48.469295 52398 api.go:53] PageSize not implemented W0909 13:48:48.469411 52398 api.go:53] PageSize not implemented I0909 13:48:48.472503 52398 executor.go:111] Tasks: 116 done / 126 total; 6 can run I0909 13:48:48.472660 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.472820 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.474346 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.474489 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.474589 52398 executor.go:111] Tasks: 122 done / 126 total; 2 can run I0909 13:48:48.474778 52398 executor.go:111] Tasks: 124 done / 126 total; 2 can run I0909 13:48:48.474903 52398 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0909 13:48:48.478591 52398 target.go:116] Terraform output is in /tmp/test4093114761/out --- PASS: TestPrivateCiliumAdvanced (0.22s) === RUN TestPrivateCiliumENI I0909 13:48:48.480983 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:48.481063 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:48.481093 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:48.481123 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:48.481147 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:48.481179 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:48.481231 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:48.481252 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:48.481281 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:48.481302 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:48.497356 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:48.499290 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.499347 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.499383 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.569990 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:48.662328 52398 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0909 13:48:48.662653 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:48.662674 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:48.663216 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:48.667878 52398 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0909 13:48:48.669862 52398 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0909 13:48:48.670234 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.670322 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.670381 52398 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0909 13:48:48.670410 52398 api.go:53] PageSize not implemented W0909 13:48:48.670552 52398 api.go:53] PageSize not implemented I0909 13:48:48.673170 52398 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0909 13:48:48.673298 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.673481 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.674333 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.674731 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.674804 52398 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0909 13:48:48.674985 52398 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0909 13:48:48.675097 52398 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0909 13:48:48.678565 52398 target.go:116] Terraform output is in /tmp/test193891489/out --- PASS: TestPrivateCiliumENI (0.20s) === RUN TestPrivateCanal I0909 13:48:48.680819 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:48.680891 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:48.680914 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:48.680941 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:48.680965 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:48.680996 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:48.681041 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:48.681058 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:48.681083 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:48.681100 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:48.696877 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:48:48.698721 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.698771 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.698807 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:48.770359 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:49.117213 52398 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0909 13:48:49.117946 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:49.117975 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:49.118048 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:49.120459 52398 executor.go:111] Tasks: 50 done / 123 total; 26 can run I0909 13:48:49.124052 52398 executor.go:111] Tasks: 76 done / 123 total; 32 can run I0909 13:48:49.124397 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.124520 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.124592 52398 executor.go:111] Tasks: 108 done / 123 total; 5 can run W0909 13:48:49.124684 52398 api.go:53] PageSize not implemented W0909 13:48:49.124716 52398 api.go:53] PageSize not implemented I0909 13:48:49.127427 52398 executor.go:111] Tasks: 113 done / 123 total; 6 can run I0909 13:48:49.127587 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.127769 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.128474 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.129249 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.129318 52398 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0909 13:48:49.129493 52398 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0909 13:48:49.129609 52398 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0909 13:48:49.133294 52398 target.go:116] Terraform output is in /tmp/test2817846156/out --- PASS: TestPrivateCanal (0.47s) === RUN TestPrivateKopeio I0909 13:48:49.150255 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:49.150337 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:49.150363 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:49.150392 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:49.150416 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:49.150447 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:49.150496 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:49.150529 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:49.150554 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:49.150571 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:49.167035 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:49.168909 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.168960 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.168996 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.246136 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:49.318832 52398 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0909 13:48:49.319377 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:49.319413 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:49.319476 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:49.320788 52398 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0909 13:48:49.322782 52398 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0909 13:48:49.323972 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.324114 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.324614 52398 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0909 13:48:49.324660 52398 api.go:53] PageSize not implemented W0909 13:48:49.324744 52398 api.go:53] PageSize not implemented I0909 13:48:49.328935 52398 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0909 13:48:49.329044 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.329227 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.329834 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.330691 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.330772 52398 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0909 13:48:49.330960 52398 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0909 13:48:49.331056 52398 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0909 13:48:49.334691 52398 target.go:116] Terraform output is in /tmp/test3338732975/out --- PASS: TestPrivateKopeio (0.19s) === RUN TestUnmanaged I0909 13:48:49.336837 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:49.336938 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:49.336972 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:49.337008 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:49.337043 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:49.337089 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:49.337129 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:49.337153 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:49.337209 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:49.337234 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:49.362729 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:49.365618 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.365699 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.365754 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.436189 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:49.504466 52398 executor.go:111] Tasks: 0 done / 110 total; 45 can run I0909 13:48:49.505277 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:49.505299 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:49.505362 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:49.507200 52398 executor.go:111] Tasks: 45 done / 110 total; 23 can run I0909 13:48:49.510616 52398 executor.go:111] Tasks: 68 done / 110 total; 29 can run I0909 13:48:49.511059 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.511154 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.511239 52398 executor.go:111] Tasks: 97 done / 110 total; 4 can run W0909 13:48:49.511299 52398 api.go:53] PageSize not implemented W0909 13:48:49.511403 52398 api.go:53] PageSize not implemented I0909 13:48:49.515947 52398 executor.go:111] Tasks: 101 done / 110 total; 5 can run I0909 13:48:49.516118 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.516265 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.517675 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.518490 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.518605 52398 executor.go:111] Tasks: 106 done / 110 total; 2 can run I0909 13:48:49.518895 52398 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0909 13:48:49.519091 52398 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0909 13:48:49.525114 52398 target.go:116] Terraform output is in /tmp/test2474261372/out --- PASS: TestUnmanaged (0.19s) === RUN TestPrivateSharedSubnet I0909 13:48:49.528738 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:49.528862 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:49.528902 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:49.528944 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:49.528989 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:49.529041 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:49.529083 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:49.529113 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:49.529152 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:49.529185 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:49.561677 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:49.564798 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.564890 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.564955 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.636123 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:49.730917 52398 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0909 13:48:49.733359 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:49.733395 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:49.733507 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:49.737929 52398 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0909 13:48:49.738282 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:49.741400 52398 executor.go:111] Tasks: 67 done / 107 total; 27 can run I0909 13:48:49.741614 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.741719 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.742087 52398 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0909 13:48:49.742144 52398 api.go:53] PageSize not implemented W0909 13:48:49.742343 52398 api.go:53] PageSize not implemented I0909 13:48:49.747114 52398 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0909 13:48:49.747302 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.747434 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.748923 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.749610 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.749724 52398 executor.go:111] Tasks: 103 done / 107 total; 2 can run I0909 13:48:49.750012 52398 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0909 13:48:49.750273 52398 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0909 13:48:49.755658 52398 target.go:116] Terraform output is in /tmp/test57830065/out --- PASS: TestPrivateSharedSubnet (0.23s) === RUN TestPrivateSharedIP I0909 13:48:49.759340 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:49.759466 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:49.759505 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:49.759559 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:49.759601 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:49.759652 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:49.759693 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:49.759723 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:49.759763 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:49.759792 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:49.788398 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:49.791236 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.791310 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.791362 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.862440 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:49.952657 52398 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0909 13:48:49.953298 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:49.953329 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:49.953424 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:49.956001 52398 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0909 13:48:49.956229 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:49.958070 52398 executor.go:111] Tasks: 69 done / 116 total; 32 can run I0909 13:48:49.958446 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.958535 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.958717 52398 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0909 13:48:49.958854 52398 api.go:53] PageSize not implemented W0909 13:48:49.958938 52398 api.go:53] PageSize not implemented I0909 13:48:49.963426 52398 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0909 13:48:49.963594 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.963773 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.965177 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.965951 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:49.966068 52398 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0909 13:48:49.966392 52398 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0909 13:48:49.966584 52398 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0909 13:48:49.976655 52398 target.go:116] Terraform output is in /tmp/test1585200007/out --- PASS: TestPrivateSharedIP (0.22s) === RUN TestPrivateDns1 I0909 13:48:49.980600 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:49.980725 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:49.980768 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:49.980817 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:49.980858 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:49.980908 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:49.980995 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:49.981028 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:49.981068 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:49.981097 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:50.012580 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.012671 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.012738 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.151936 52398 executor.go:111] Tasks: 0 done / 120 total; 46 can run I0909 13:48:50.155421 52398 executor.go:111] Tasks: 46 done / 120 total; 24 can run I0909 13:48:50.155880 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "internal.example.com" I0909 13:48:50.155937 52398 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0909 13:48:50.156068 52398 dnszone.go:248] Existing zone "internal.example.com." found; will configure TF to reuse I0909 13:48:50.156091 52398 dnszone.go:271] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0909 13:48:50.156185 52398 executor.go:111] Tasks: 70 done / 120 total; 35 can run I0909 13:48:50.156653 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.156836 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.158128 52398 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0909 13:48:50.158164 52398 api.go:53] PageSize not implemented W0909 13:48:50.158339 52398 api.go:53] PageSize not implemented I0909 13:48:50.162927 52398 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0909 13:48:50.163161 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.163337 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.164584 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.164739 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.164849 52398 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0909 13:48:50.165127 52398 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0909 13:48:50.165262 52398 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0909 13:48:50.176114 52398 target.go:116] Terraform output is in /tmp/test2880257003/out --- PASS: TestPrivateDns1 (0.20s) === RUN TestPrivateDns2 I0909 13:48:50.180297 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:50.180422 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:50.180464 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:50.180530 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:50.180578 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:50.180630 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:50.180671 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:50.180700 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:50.180739 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:50.180766 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:50.212104 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.212190 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.212256 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.349836 52398 executor.go:111] Tasks: 0 done / 116 total; 44 can run I0909 13:48:50.354264 52398 executor.go:111] Tasks: 44 done / 116 total; 22 can run I0909 13:48:50.355688 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "private.example.com" I0909 13:48:50.355688 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:50.355818 52398 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0909 13:48:50.356437 52398 dnszone.go:248] Existing zone "private.example.com." found; will configure TF to reuse I0909 13:48:50.356470 52398 dnszone.go:262] VPC "vpc-12345678" already associated with zone "private.example.com." I0909 13:48:50.356777 52398 executor.go:111] Tasks: 66 done / 116 total; 35 can run I0909 13:48:50.357850 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.358004 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.359673 52398 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0909 13:48:50.359784 52398 api.go:53] PageSize not implemented W0909 13:48:50.359930 52398 api.go:53] PageSize not implemented I0909 13:48:50.364744 52398 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0909 13:48:50.364916 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.365159 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.366112 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.367328 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.367448 52398 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0909 13:48:50.367726 52398 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0909 13:48:50.367892 52398 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0909 13:48:50.373568 52398 target.go:116] Terraform output is in /tmp/test2198944128/out --- PASS: TestPrivateDns2 (0.20s) === RUN TestDiscoveryFeatureGate I0909 13:48:50.377199 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:50.377323 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:50.377361 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:50.377441 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:50.377482 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:50.377532 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:50.377572 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:50.377601 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:50.377636 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:50.377662 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:50.403200 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:48:50.406097 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.406178 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.478040 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:50.646571 52398 executor.go:111] Tasks: 0 done / 99 total; 51 can run I0909 13:48:50.647330 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:50.647367 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:50.647459 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:50.650472 52398 executor.go:111] Tasks: 51 done / 99 total; 23 can run I0909 13:48:50.658216 52398 executor.go:111] Tasks: 74 done / 99 total; 21 can run I0909 13:48:50.658380 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.658694 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.659355 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.660188 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.660263 52398 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0909 13:48:50.660433 52398 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0909 13:48:50.660600 52398 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0909 13:48:50.664013 52398 target.go:116] Terraform output is in /tmp/test1543815798/out --- PASS: TestDiscoveryFeatureGate (0.29s) === RUN TestVFSServiceAccountIssuerDiscovery I0909 13:48:50.666695 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:50.666779 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:50.666806 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:50.666834 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:50.666863 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:50.666896 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:50.666925 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:50.666945 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:50.666970 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:50.666989 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:50.681286 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:50.683024 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.683082 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.762010 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:50.829215 52398 executor.go:111] Tasks: 0 done / 90 total; 45 can run I0909 13:48:50.829944 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:50.829974 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:50.830067 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:50.832500 52398 executor.go:111] Tasks: 45 done / 90 total; 20 can run I0909 13:48:50.837703 52398 executor.go:111] Tasks: 65 done / 90 total; 21 can run I0909 13:48:50.837966 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.838259 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.839391 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.839829 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.839968 52398 executor.go:111] Tasks: 86 done / 90 total; 2 can run I0909 13:48:50.840250 52398 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0909 13:48:50.840417 52398 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0909 13:48:50.845011 52398 target.go:116] Terraform output is in /tmp/test1938442472/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.18s) === RUN TestAWSLBController I0909 13:48:50.848703 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:50.848857 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:50.848909 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:50.848980 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:50.849045 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:50.849135 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:50.849236 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:50.849294 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:50.849362 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:50.849410 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:50.875739 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:50.877708 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.877772 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:50.948960 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:51.530194 52398 executor.go:111] Tasks: 0 done / 96 total; 49 can run I0909 13:48:51.530552 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:51.530580 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:51.530672 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:51.535557 52398 executor.go:111] Tasks: 49 done / 96 total; 22 can run I0909 13:48:51.539522 52398 executor.go:111] Tasks: 71 done / 96 total; 21 can run I0909 13:48:51.539958 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:51.540165 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:51.540925 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:51.543062 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:51.543191 52398 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0909 13:48:51.543439 52398 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0909 13:48:51.543574 52398 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0909 13:48:51.549899 52398 target.go:116] Terraform output is in /tmp/test2838872838/out --- PASS: TestAWSLBController (0.71s) === RUN TestManyAddons I0909 13:48:51.558825 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:51.558991 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:51.559042 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:51.559091 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:51.559132 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:51.559188 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:51.559281 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:51.559314 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:51.559352 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:51.559382 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:51.584893 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:51.587732 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:51.587811 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:51.661666 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:52.438154 52398 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0909 13:48:52.443047 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:52.443109 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:52.443675 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:52.446017 52398 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0909 13:48:52.452228 52398 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0909 13:48:52.452790 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:52.453021 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:52.454359 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:52.455660 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:52.455779 52398 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0909 13:48:52.456086 52398 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0909 13:48:52.456354 52398 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0909 13:48:52.462597 52398 target.go:116] Terraform output is in /tmp/test2337160758/out --- PASS: TestManyAddons (0.91s) === RUN TestManyAddonsCCMIRSA I0909 13:48:52.472302 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:52.472404 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:52.472436 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:52.472467 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:52.472494 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:52.472529 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:52.472585 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:52.472602 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:52.472626 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:52.472643 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:52.489283 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:52.491570 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:52.491626 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:52.563860 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:53.316444 52398 executor.go:111] Tasks: 0 done / 123 total; 61 can run I0909 13:48:53.317118 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:53.317197 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:53.318571 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:53.324784 52398 executor.go:111] Tasks: 61 done / 123 total; 30 can run I0909 13:48:53.330190 52398 executor.go:111] Tasks: 91 done / 123 total; 26 can run I0909 13:48:53.330626 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:53.330896 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:53.333036 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:53.334370 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:53.334503 52398 executor.go:111] Tasks: 117 done / 123 total; 2 can run I0909 13:48:53.334879 52398 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0909 13:48:53.335214 52398 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0909 13:48:53.342851 52398 target.go:116] Terraform output is in /tmp/test1091495379/out --- PASS: TestManyAddonsCCMIRSA (0.88s) === RUN TestManyAddonsCCMIRSA23 I0909 13:48:53.357055 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:53.357349 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:53.357439 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:53.357541 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:53.357625 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:53.357730 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:53.357861 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:53.357894 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:53.357932 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:53.357962 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:53.386431 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0909 13:48:53.389423 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:53.389494 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:53.462844 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:54.213512 52398 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0909 13:48:54.214180 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:54.214220 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:54.214314 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:54.216446 52398 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0909 13:48:54.220925 52398 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0909 13:48:54.221158 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:54.221365 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:54.222513 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:54.223600 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:54.223732 52398 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0909 13:48:54.223992 52398 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0909 13:48:54.224341 52398 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0909 13:48:54.244114 52398 target.go:116] Terraform output is in /tmp/test1509180226/out --- PASS: TestManyAddonsCCMIRSA23 (0.89s) === RUN TestManyAddonsCCMIRSA24 I0909 13:48:54.250635 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:54.250738 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:54.250771 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:54.250816 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:54.250844 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:54.250876 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:54.250930 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:54.250950 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:54.250974 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:54.250991 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:54.278120 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:48:54.281505 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:54.281597 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:54.352869 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:55.090872 52398 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0909 13:48:55.093879 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:55.093944 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:55.094046 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:55.099316 52398 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0909 13:48:55.104258 52398 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0909 13:48:55.104559 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:55.104738 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:55.105573 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:55.107440 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:55.107563 52398 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0909 13:48:55.107803 52398 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0909 13:48:55.108035 52398 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0909 13:48:55.119013 52398 target.go:116] Terraform output is in /tmp/test516593182/out --- PASS: TestManyAddonsCCMIRSA24 (0.88s) === RUN TestManyAddonsCCMIRSA25 I0909 13:48:55.127782 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:55.127966 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:55.128049 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:55.128142 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:55.128224 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:55.128313 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:55.128442 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:55.128514 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:55.128593 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:55.128659 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:55.151589 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0909 13:48:55.153727 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:55.153794 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:55.230118 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:55.948647 52398 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0909 13:48:55.949161 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:55.949187 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:55.949247 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:55.951185 52398 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0909 13:48:55.956258 52398 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0909 13:48:55.956526 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:55.956722 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:55.958050 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:55.958780 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:55.958864 52398 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0909 13:48:55.959108 52398 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0909 13:48:55.959309 52398 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0909 13:48:55.967486 52398 target.go:116] Terraform output is in /tmp/test3153982613/out --- PASS: TestManyAddonsCCMIRSA25 (0.85s) === RUN TestManyAddonsCCMIRSA26 I0909 13:48:55.974240 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:55.974338 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:55.974369 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:55.974431 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:55.974458 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:55.974492 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:55.974519 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:55.974538 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:55.974566 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:55.974584 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:55.991187 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0909 13:48:55.993263 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:55.993321 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:56.082916 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:56.756803 52398 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0909 13:48:56.757302 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:56.757339 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:56.757428 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:56.759195 52398 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0909 13:48:56.766146 52398 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0909 13:48:56.766550 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:56.766744 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:56.768136 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:56.768249 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:56.768360 52398 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0909 13:48:56.768593 52398 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0909 13:48:56.768796 52398 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0909 13:48:56.773714 52398 target.go:116] Terraform output is in /tmp/test3841260474/out --- PASS: TestManyAddonsCCMIRSA26 (0.81s) === RUN TestManyAddonsGCE I0909 13:48:56.779613 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0909 13:48:57.453626 52398 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0909 13:48:57.453730 52398 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0909 13:48:57.453869 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com/backups/etcd/main" W0909 13:48:57.453899 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com/backups/etcd/events" W0909 13:48:57.453926 52398 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com" W0909 13:48:57.453995 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0909 13:48:57.454946 52398 executor.go:111] Tasks: 0 done / 67 total; 44 can run I0909 13:48:57.462143 52398 executor.go:111] Tasks: 44 done / 67 total; 17 can run I0909 13:48:57.465201 52398 executor.go:111] Tasks: 61 done / 67 total; 4 can run I0909 13:48:57.466771 52398 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0909 13:48:57.466880 52398 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0909 13:48:57.469167 52398 target.go:116] Terraform output is in /tmp/test442368518/out --- PASS: TestManyAddonsGCE (0.69s) === RUN TestCCM I0909 13:48:57.473231 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:57.473312 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:57.473343 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:57.473400 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:57.473428 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:57.473461 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:57.473488 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:57.473506 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:57.473530 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:57.473548 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:57.488942 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:57.490985 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:57.491039 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:57.563638 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:58.183929 52398 executor.go:111] Tasks: 0 done / 110 total; 53 can run I0909 13:48:58.184400 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:58.184431 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:58.184518 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:58.190702 52398 executor.go:111] Tasks: 53 done / 110 total; 24 can run I0909 13:48:58.194818 52398 executor.go:111] Tasks: 77 done / 110 total; 27 can run I0909 13:48:58.194980 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.195247 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.196136 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.196520 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.196607 52398 executor.go:111] Tasks: 104 done / 110 total; 2 can run I0909 13:48:58.196786 52398 executor.go:111] Tasks: 106 done / 110 total; 4 can run I0909 13:48:58.196951 52398 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0909 13:48:58.202474 52398 target.go:116] Terraform output is in /tmp/test829326763/out --- PASS: TestCCM (0.74s) === RUN TestExternalDNS I0909 13:48:58.210828 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:58.210917 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:58.210944 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:58.210973 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:58.210997 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:58.211028 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:58.211055 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:58.211073 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:58.211096 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:58.211113 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:58.225679 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:58.227295 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.227346 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.298662 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:58.362268 52398 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0909 13:48:58.362539 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:58.362561 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:58.362650 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:58.367814 52398 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0909 13:48:58.370838 52398 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0909 13:48:58.371158 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.371328 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.372057 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.372705 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.372790 52398 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0909 13:48:58.372968 52398 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0909 13:48:58.373066 52398 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0909 13:48:58.375821 52398 target.go:116] Terraform output is in /tmp/test3441949750/out --- PASS: TestExternalDNS (0.17s) === RUN TestExternalDNSIRSA I0909 13:48:58.377812 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:58.377899 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:58.377930 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:58.377978 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:58.378013 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:58.378062 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:58.378098 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:58.378128 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:58.378168 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:58.378193 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:58.392639 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:58.394354 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.394416 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.464725 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:58.560953 52398 executor.go:111] Tasks: 0 done / 92 total; 46 can run I0909 13:48:58.561550 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:58.561596 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:58.561693 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:58.563675 52398 executor.go:111] Tasks: 46 done / 92 total; 21 can run I0909 13:48:58.568423 52398 executor.go:111] Tasks: 67 done / 92 total; 21 can run I0909 13:48:58.568893 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.569174 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.570340 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.571603 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.571720 52398 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0909 13:48:58.572001 52398 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0909 13:48:58.572172 52398 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0909 13:48:58.581091 52398 target.go:116] Terraform output is in /tmp/test2949787872/out --- PASS: TestExternalDNSIRSA (0.21s) === RUN TestKarpenter I0909 13:48:58.585214 52398 featureflag.go:166] FeatureFlag "Karpenter"=true I0909 13:48:58.585256 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:58.585378 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:58.585420 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:58.585521 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:58.585566 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:58.585615 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:58.585656 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:58.585687 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:58.585724 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:58.585752 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:58.616832 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:48:58.620232 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.620317 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.620377 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.702395 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:58.877108 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.877272 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.877388 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.969160 52398 executor.go:111] Tasks: 0 done / 108 total; 53 can run I0909 13:48:58.969929 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:58.969961 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:58.970047 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:58.971493 52398 executor.go:111] Tasks: 53 done / 108 total; 26 can run I0909 13:48:58.978208 52398 executor.go:111] Tasks: 79 done / 108 total; 25 can run I0909 13:48:58.978639 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.978875 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.979181 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.979469 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.980279 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.980539 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.981954 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.983286 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:58.983426 52398 executor.go:111] Tasks: 104 done / 108 total; 2 can run I0909 13:48:58.983791 52398 executor.go:111] Tasks: 106 done / 108 total; 2 can run I0909 13:48:58.984019 52398 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0909 13:48:58.990399 52398 target.go:116] Terraform output is in /tmp/test1184811060/out I0909 13:48:58.995950 52398 featureflag.go:166] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.41s) === RUN TestSharedSubnet I0909 13:48:58.996097 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:58.996228 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:58.996269 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:58.996314 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:58.996353 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:58.996405 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:58.996447 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:58.996481 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:58.996518 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:58.996546 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:59.019473 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:59.021161 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.021219 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.094708 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:59.186314 52398 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0909 13:48:59.186753 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:59.186789 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:59.188537 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:59.193807 52398 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0909 13:48:59.194207 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:59.198743 52398 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0909 13:48:59.199164 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.199398 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.200716 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.201863 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.201976 52398 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0909 13:48:59.202268 52398 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0909 13:48:59.202453 52398 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0909 13:48:59.206601 52398 target.go:116] Terraform output is in /tmp/test3746830097/out --- PASS: TestSharedSubnet (0.23s) === RUN TestSharedVPC I0909 13:48:59.221390 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:59.221520 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:59.221566 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:59.221612 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:59.221651 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:59.221702 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:59.221748 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:59.221777 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:59.221816 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:59.221845 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:59.246642 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:59.249532 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.249618 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.321131 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:59.396656 52398 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0909 13:48:59.397138 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:59.397181 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:59.397274 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:59.399895 52398 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0909 13:48:59.400273 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:59.406536 52398 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0909 13:48:59.407013 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.407278 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.408200 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.408992 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.409066 52398 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0909 13:48:59.409305 52398 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0909 13:48:59.409430 52398 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0909 13:48:59.413309 52398 target.go:116] Terraform output is in /tmp/test2516853608/out --- PASS: TestSharedVPC (0.19s) === RUN TestSharedVPCIPv6 I0909 13:48:59.416444 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:59.416570 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:59.416616 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:59.416664 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:59.416706 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:59.416754 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:59.416832 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:59.416863 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:59.416902 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:59.416930 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:59.418841 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:59.418883 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:59.435759 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:48:59.435840 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:59.435977 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:48:59.440756 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.440825 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.512275 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:59.679069 52398 executor.go:111] Tasks: 0 done / 121 total; 45 can run I0909 13:48:59.679441 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:59.679464 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:59.679534 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:59.685810 52398 executor.go:111] Tasks: 45 done / 121 total; 25 can run I0909 13:48:59.686346 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:59.686461 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:48:59.689198 52398 executor.go:111] Tasks: 70 done / 121 total; 33 can run I0909 13:48:59.689864 52398 executor.go:111] Tasks: 103 done / 121 total; 6 can run I0909 13:48:59.690037 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:59.690103 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:59.690231 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:48:59.690321 52398 loadbalancers.go:35] PageSize not implemented I0909 13:48:59.695323 52398 executor.go:111] Tasks: 109 done / 121 total; 8 can run I0909 13:48:59.695522 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.695666 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.697233 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.698215 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.698335 52398 executor.go:111] Tasks: 117 done / 121 total; 2 can run I0909 13:48:59.698635 52398 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0909 13:48:59.698883 52398 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0909 13:48:59.704643 52398 target.go:116] Terraform output is in /tmp/test3944818185/out --- PASS: TestSharedVPCIPv6 (0.29s) === RUN TestExistingIAM I0909 13:48:59.708570 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:59.708698 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:59.708742 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:59.708786 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:59.708828 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:59.708877 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:59.708923 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:59.708955 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:59.708994 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:59.709025 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:59.741356 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:59.744237 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.744309 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.744358 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.744405 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.819642 52398 zones.go:93] ListHostedZonesPages { } I0909 13:48:59.908171 52398 executor.go:111] Tasks: 0 done / 101 total; 51 can run I0909 13:48:59.908530 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:48:59.908564 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:48:59.908662 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:48:59.912788 52398 executor.go:111] Tasks: 51 done / 101 total; 15 can run I0909 13:48:59.917295 52398 executor.go:111] Tasks: 66 done / 101 total; 27 can run I0909 13:48:59.917437 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.917611 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.917730 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.917834 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.918464 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.918811 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.919048 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.919168 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.919248 52398 executor.go:111] Tasks: 93 done / 101 total; 4 can run I0909 13:48:59.919517 52398 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0909 13:48:59.919687 52398 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0909 13:48:59.923379 52398 target.go:116] Terraform output is in /tmp/test4147901431/out --- PASS: TestExistingIAM (0.22s) === RUN TestPhaseNetwork I0909 13:48:59.925819 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:59.925899 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:59.925926 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:59.925955 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:59.925979 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:59.926012 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:59.926037 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:59.926058 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:59.926081 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:59.926098 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:59.945188 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:48:59.947000 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.947060 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:48:59.947097 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.019406 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:00.087577 52398 executor.go:111] Tasks: 0 done / 119 total; 46 can run I0909 13:49:00.087952 52398 executor.go:111] Tasks: 46 done / 119 total; 26 can run I0909 13:49:00.088306 52398 executor.go:111] Tasks: 72 done / 119 total; 32 can run I0909 13:49:00.088626 52398 executor.go:111] Tasks: 104 done / 119 total; 5 can run I0909 13:49:00.088739 52398 executor.go:111] Tasks: 109 done / 119 total; 6 can run I0909 13:49:00.088817 52398 executor.go:111] Tasks: 115 done / 119 total; 2 can run I0909 13:49:00.088848 52398 executor.go:111] Tasks: 117 done / 119 total; 2 can run I0909 13:49:00.088867 52398 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0909 13:49:00.089481 52398 target.go:116] Terraform output is in /tmp/test1474106910/out --- PASS: TestPhaseNetwork (0.16s) === RUN TestExternalLoadBalancer I0909 13:49:00.089851 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:00.089929 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:00.089954 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:00.089982 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:00.090010 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:00.090042 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:00.090077 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:00.090096 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:00.090120 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:00.090140 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:00.108354 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:00.110044 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.110105 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.181073 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:00.249499 52398 executor.go:111] Tasks: 0 done / 93 total; 49 can run I0909 13:49:00.249904 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:00.249928 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:00.250025 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:00.254423 52398 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0909 13:49:00.259475 52398 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0909 13:49:00.259659 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.259905 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.260763 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.261098 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.261169 52398 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0909 13:49:00.261352 52398 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0909 13:49:00.261464 52398 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0909 13:49:00.264009 52398 target.go:116] Terraform output is in /tmp/test4196585018/out --- PASS: TestExternalLoadBalancer (0.18s) === RUN TestPhaseIAM integration_test.go:920: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:928: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0909 13:49:00.266103 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:00.266180 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:00.266206 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:00.266237 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:00.266261 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:00.266293 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:00.266320 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:00.266337 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:00.266360 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:00.266377 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:00.285833 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:00.287843 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.287902 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.287938 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.287972 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.288003 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.288031 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.288061 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.359470 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:00.432864 52398 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0909 13:49:00.433259 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:00.433283 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:00.433348 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:00.435274 52398 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0909 13:49:00.441836 52398 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0909 13:49:00.442017 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.442231 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.442374 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.442536 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.442914 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.443575 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.444586 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.444792 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.444874 52398 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0909 13:49:00.445127 52398 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0909 13:49:00.445317 52398 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0909 13:49:00.449079 52398 target.go:116] Terraform output is in /tmp/test3300228568/out --- PASS: TestMixedInstancesASG (0.19s) === RUN TestMixedInstancesSpotASG I0909 13:49:00.451701 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:00.451784 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:00.451812 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:00.451840 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:00.451864 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:00.451897 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:00.451923 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:00.451941 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:00.451966 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:00.451983 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:00.474385 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:00.478494 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.478596 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.478660 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.478716 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.478769 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.478817 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.478862 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.550791 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:00.619998 52398 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0909 13:49:00.622575 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:00.622607 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:00.622707 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:00.627092 52398 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0909 13:49:00.632298 52398 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0909 13:49:00.632650 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.632807 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.632922 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.633078 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.633818 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.634379 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.636503 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.636936 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.637063 52398 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0909 13:49:00.637448 52398 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0909 13:49:00.637787 52398 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0909 13:49:00.644495 52398 target.go:116] Terraform output is in /tmp/test1637305032/out --- PASS: TestMixedInstancesSpotASG (0.20s) === RUN TestAdditionalObjects I0909 13:49:00.649491 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:00.649615 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:00.649654 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:00.649703 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:00.649743 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:00.649791 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:00.649872 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:00.649903 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:00.649942 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:00.649970 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:00.674187 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0909 13:49:00.676047 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.676109 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.748166 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:00.874863 52398 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0909 13:49:00.875089 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:00.875115 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:00.875204 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:00.876829 52398 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0909 13:49:00.882837 52398 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0909 13:49:00.883146 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.883323 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.884076 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.884914 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.884987 52398 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0909 13:49:00.885268 52398 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0909 13:49:00.885456 52398 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0909 13:49:00.888246 52398 target.go:116] Terraform output is in /tmp/test657495350/out --- PASS: TestAdditionalObjects (0.24s) === RUN TestContainerd I0909 13:49:00.890397 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:00.890473 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:00.890497 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:00.890540 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:00.890564 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:00.890594 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:00.890621 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:00.890637 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:00.890660 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:00.890677 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:00.906992 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:00.909159 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.909227 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:00.980367 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:01.059465 52398 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0909 13:49:01.059775 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:01.059825 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:01.059942 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:01.061674 52398 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0909 13:49:01.064863 52398 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0909 13:49:01.065135 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.065234 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.066060 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.066833 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.066929 52398 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0909 13:49:01.067119 52398 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0909 13:49:01.067279 52398 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0909 13:49:01.069793 52398 target.go:116] Terraform output is in /tmp/test3717639106/out --- PASS: TestContainerd (0.18s) === RUN TestContainerdCustom I0909 13:49:01.071782 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:01.071863 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:01.071890 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:01.071919 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:01.071945 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:01.071978 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:01.072004 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:01.072022 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:01.072046 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:01.072063 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:01.091386 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:01.093150 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.093208 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.163668 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:01.226940 52398 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0909 13:49:01.227262 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:01.227288 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:01.227403 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:01.232371 52398 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0909 13:49:01.235358 52398 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0909 13:49:01.235569 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.235734 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.236700 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.236956 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.237029 52398 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0909 13:49:01.237221 52398 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0909 13:49:01.237334 52398 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0909 13:49:01.239836 52398 target.go:116] Terraform output is in /tmp/test1532869010/out --- PASS: TestContainerdCustom (0.17s) === RUN TestDockerCustom I0909 13:49:01.241840 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:01.241913 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:01.241937 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:01.241962 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:01.241984 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:01.242017 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:01.242044 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:01.242061 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:01.242084 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:01.242102 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:01.256783 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:01.258600 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.258652 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.328855 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:01.392933 52398 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0909 13:49:01.393358 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:01.393378 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:01.393461 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:01.395183 52398 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0909 13:49:01.398559 52398 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0909 13:49:01.398690 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.398855 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.399811 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.400565 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.400638 52398 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0909 13:49:01.400893 52398 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0909 13:49:01.401014 52398 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0909 13:49:01.403620 52398 target.go:116] Terraform output is in /tmp/test4025137988/out --- PASS: TestDockerCustom (0.16s) === RUN TestAPIServerNodes I0909 13:49:01.405620 52398 featureflag.go:166] FeatureFlag "APIServerNodes"=true I0909 13:49:01.405639 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:01.405714 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:01.405739 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:01.405766 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:01.405793 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:01.405827 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:01.405875 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:01.405893 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:01.405916 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:01.405933 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:01.423474 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.17 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.17 ********************************************************************************* I0909 13:49:01.425266 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.425323 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:01.425359 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.152027 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:02.321812 52398 executor.go:111] Tasks: 0 done / 100 total; 47 can run I0909 13:49:02.322771 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:02.322808 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:02.322897 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:02.324237 52398 executor.go:111] Tasks: 47 done / 100 total; 24 can run I0909 13:49:02.330163 52398 executor.go:111] Tasks: 71 done / 100 total; 23 can run I0909 13:49:02.330404 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.330589 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.330778 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.331321 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.331572 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.333539 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.333670 52398 executor.go:111] Tasks: 94 done / 100 total; 3 can run I0909 13:49:02.334128 52398 executor.go:111] Tasks: 97 done / 100 total; 3 can run I0909 13:49:02.334323 52398 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0909 13:49:02.339661 52398 target.go:116] Terraform output is in /tmp/test1703522803/out I0909 13:49:02.343993 52398 featureflag.go:166] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.94s) === RUN TestNTHIMDSProcessor I0909 13:49:02.344133 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.344276 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:02.344322 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:02.344376 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:02.344420 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:02.344474 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:02.344567 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:02.344602 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:02.344642 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:02.344678 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:02.367548 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:02.369605 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.369670 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.446814 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:02.533002 52398 executor.go:111] Tasks: 0 done / 88 total; 44 can run I0909 13:49:02.533467 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:02.533502 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:02.533592 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:02.535104 52398 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0909 13:49:02.539411 52398 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0909 13:49:02.539734 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.539932 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.541122 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.542304 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.542415 52398 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0909 13:49:02.542676 52398 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0909 13:49:02.542844 52398 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0909 13:49:02.545549 52398 target.go:116] Terraform output is in /tmp/test1043777428/out --- PASS: TestNTHIMDSProcessor (0.20s) === RUN TestNTHIMDSProcessorIRSA I0909 13:49:02.547813 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.547918 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:02.547949 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:02.547984 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:02.548009 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:02.548042 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:02.548097 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:02.548116 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:02.548144 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:02.548162 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:02.566218 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:02.568077 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.568135 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.640399 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:02.783217 52398 executor.go:111] Tasks: 0 done / 95 total; 48 can run I0909 13:49:02.784114 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:02.784142 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:02.784253 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:02.785459 52398 executor.go:111] Tasks: 48 done / 95 total; 22 can run I0909 13:49:02.792234 52398 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0909 13:49:02.792554 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.792752 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.793767 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.794366 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.794448 52398 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0909 13:49:02.794761 52398 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0909 13:49:02.794972 52398 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0909 13:49:02.798412 52398 target.go:116] Terraform output is in /tmp/test416569451/out --- PASS: TestNTHIMDSProcessorIRSA (0.25s) === RUN TestCustomIRSA I0909 13:49:02.800903 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.800990 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:02.801023 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:02.801054 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:02.801078 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:02.801111 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:02.801165 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:02.801186 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:02.801210 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:02.801227 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:02.817869 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:02.819844 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.819905 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:02.907572 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:03.489387 52398 executor.go:111] Tasks: 0 done / 100 total; 50 can run I0909 13:49:03.489891 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:03.489929 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:03.490018 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:03.496896 52398 executor.go:111] Tasks: 50 done / 100 total; 25 can run I0909 13:49:03.500930 52398 executor.go:111] Tasks: 75 done / 100 total; 21 can run I0909 13:49:03.501109 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.501307 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.501993 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.504037 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.504259 52398 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0909 13:49:03.504792 52398 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0909 13:49:03.505016 52398 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0909 13:49:03.511754 52398 target.go:116] Terraform output is in /tmp/test1326518351/out --- PASS: TestCustomIRSA (0.72s) === RUN TestClusterNameDigit I0909 13:49:03.518128 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:03.518348 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:03.518425 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:03.518527 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:03.518609 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:03.518663 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:03.518710 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:03.518743 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:03.518784 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:03.518813 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:03.546332 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:03.549492 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.549578 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.620362 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:03.714410 52398 executor.go:111] Tasks: 0 done / 95 total; 47 can run I0909 13:49:03.715166 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:03.715198 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:03.715285 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:03.718221 52398 executor.go:111] Tasks: 47 done / 95 total; 23 can run I0909 13:49:03.725112 52398 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0909 13:49:03.725378 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.725544 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.726377 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.727159 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.727252 52398 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0909 13:49:03.727408 52398 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0909 13:49:03.727602 52398 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0909 13:49:03.730702 52398 target.go:116] Terraform output is in /tmp/test1488312307/out --- PASS: TestClusterNameDigit (0.21s) === RUN TestCASPriorityExpander I0909 13:49:03.732995 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:03.733071 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:03.733096 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:03.733125 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:03.733149 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:03.733181 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:03.733230 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:03.733250 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:03.733273 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:03.733291 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:03.750403 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0909 13:49:03.752411 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.752470 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.752503 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:03.752538 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0909 13:49:04.467567 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:04.590042 52398 executor.go:111] Tasks: 0 done / 102 total; 48 can run I0909 13:49:04.590709 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:04.590753 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:04.590854 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:04.592184 52398 executor.go:111] Tasks: 48 done / 102 total; 21 can run I0909 13:49:04.597250 52398 executor.go:111] Tasks: 69 done / 102 total; 25 can run I0909 13:49:04.597508 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.597615 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.597712 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.597875 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.598503 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.598927 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.599137 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.599327 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.599433 52398 executor.go:111] Tasks: 94 done / 102 total; 4 can run I0909 13:49:04.599699 52398 executor.go:111] Tasks: 98 done / 102 total; 4 can run I0909 13:49:04.599913 52398 executor.go:111] Tasks: 102 done / 102 total; 0 can run I0909 13:49:04.605458 52398 target.go:116] Terraform output is in /tmp/test3804287266/out --- PASS: TestCASPriorityExpander (0.88s) === RUN TestCASPriorityExpanderCustom I0909 13:49:04.609817 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:04.609944 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:04.609988 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:04.610061 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:04.610106 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:04.610157 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:04.610247 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:04.610287 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:04.610335 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:04.610372 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:04.648080 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0909 13:49:04.651977 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.652071 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.652133 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.652194 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0909 13:49:04.736221 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:04.880145 52398 executor.go:111] Tasks: 0 done / 102 total; 48 can run I0909 13:49:04.883245 52398 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0909 13:49:04.883365 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:04.883472 52398 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0909 13:49:04.885990 52398 executor.go:111] Tasks: 48 done / 102 total; 21 can run I0909 13:49:04.896846 52398 executor.go:111] Tasks: 69 done / 102 total; 25 can run I0909 13:49:04.897143 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.897484 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.897686 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.899393 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.901287 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.901534 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.901651 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.903661 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.903758 52398 executor.go:111] Tasks: 94 done / 102 total; 4 can run I0909 13:49:04.904126 52398 executor.go:111] Tasks: 98 done / 102 total; 4 can run I0909 13:49:04.904313 52398 executor.go:111] Tasks: 102 done / 102 total; 0 can run I0909 13:49:04.907765 52398 target.go:116] Terraform output is in /tmp/test4207790999/out --- PASS: TestCASPriorityExpanderCustom (0.30s) === RUN TestLifecycleMinimalAWS I0909 13:49:04.910521 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:04.910618 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:04.910655 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:04.910886 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:04.910930 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:04.910979 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:04.911019 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:04.911047 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:04.911083 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:04.911108 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:04.911158 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0909 13:49:04.935838 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:04.938628 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:04.938714 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0909 13:49:05.010629 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:05.082047 52398 iamrole.go:89] ListRoles: { } I0909 13:49:05.083496 52398 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0909 13:49:05.085983 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:05.085939 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.086192 52398 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:05.086312 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0909 13:49:05.086360 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0909 13:49:05.086394 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:05.086440 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:05.086475 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:05.086582 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0909 13:49:05.086639 52398 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0909 13:49:05.086666 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0909 13:49:05.086780 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.086868 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.094176 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0909 13:49:05.094319 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" }] } I0909 13:49:05.087007 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.094644 52398 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:05.094847 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:05.094909 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:05.095191 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:05.095263 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:05.095468 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:05.096250 52398 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0909 13:49:05.096622 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0909 13:49:05.096653 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0909 13:49:05.096829 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0909 13:49:05.096952 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0909 13:49:05.097068 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:05.097308 52398 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0909 13:49:05.097368 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:05.097515 52398 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0909 13:49:05.097576 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.097683 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0909 13:49:05.097701 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0909 13:49:05.097791 52398 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0909 13:49:05.097828 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0909 13:49:05.097933 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:05.097965 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0909 13:49:05.097977 52398 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] } I0909 13:49:05.098070 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.098265 52398 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:05.098358 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.098594 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:05.098643 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0909 13:49:05.098730 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:05.099592 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:05.099686 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:05.099737 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:05.099793 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:05.099840 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:05.099920 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:05.099972 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:05.100138 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0909 13:49:05.102165 52398 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0909 13:49:05.102296 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:05.102459 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0909 13:49:05.102526 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0909 13:49:05.102691 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" }] }] } I0909 13:49:05.102824 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" }] }] } I0909 13:49:05.102952 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:05.103074 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" }] }] } I0909 13:49:05.103193 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:05.103310 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:05.103428 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:05.103506 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:05.103593 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0909 13:49:05.103620 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0909 13:49:05.103647 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:05.103732 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:05.103790 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" }] }] } I0909 13:49:05.103890 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:05.103966 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:05.104047 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0909 13:49:05.104129 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:05.104208 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:05.104257 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:05.104335 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0909 13:49:05.104398 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:05.104538 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:05.105687 52398 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0909 13:49:05.105989 52398 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0909 13:49:05.106102 52398 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0909 13:49:05.106140 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:05.106174 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:05.106241 52398 dns.go:232] Pre-creating DNS records I0909 13:49:05.106269 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:05.111131 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:05.113461 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:05.113545 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0909 13:49:05.185537 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:05.248449 52398 iamrole.go:89] ListRoles: { } I0909 13:49:05.249267 52398 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0909 13:49:05.249546 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.249688 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0909 13:49:05.249797 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:05.249761 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:05.249829 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:05.249856 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:05.249961 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.250055 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.250130 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:05.251378 52398 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0909 13:49:05.251525 52398 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0909 13:49:05.251480 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.251614 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0909 13:49:05.251572 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0909 13:49:05.251689 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:05.251741 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.251835 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0909 13:49:05.251937 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:05.251980 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:05.252031 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:05.252075 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:05.252103 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.255645 52398 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0909 13:49:05.255815 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:05.256085 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:05.256256 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:05.256524 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:05.256700 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:05.256922 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:05.257198 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:05.259670 52398 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0909 13:49:05.259831 52398 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0909 13:49:05.259937 52398 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0909 13:49:05.261413 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0909 13:49:05.261451 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:05.261476 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0909 13:49:05.261495 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:05.261514 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0909 13:49:05.261534 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0909 13:49:05.261554 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:05.261574 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0909 13:49:05.261594 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0909 13:49:05.261615 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0909 13:49:05.261639 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:05.261657 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:05.261675 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:05.261694 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0909 13:49:05.261712 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:05.261731 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:05.261750 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:05.261773 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:05.261791 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:05.261809 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0909 13:49:05.261827 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:05.261849 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:05.261871 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:05.261892 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0909 13:49:05.262859 52398 delete_cluster.go:128] Looking for cloud resources to delete W0909 13:49:05.262948 52398 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0909 13:49:05.262972 52398 keypairs.go:95] DescribeKeyPairs: { } I0909 13:49:05.263014 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.263046 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0909 13:49:05.263080 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.263117 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.263145 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.263176 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.263203 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.263235 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0909 13:49:05.263266 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.263302 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:05.263325 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0909 13:49:05.263349 52398 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0909 13:49:05.263373 52398 api.go:53] PageSize not implemented I0909 13:49:05.263387 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:05.263402 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:05.263414 52398 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0909 13:49:05.263431 52398 targetgroups.go:35] PageSize not implemented I0909 13:49:05.263450 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0909 13:49:05.263486 52398 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0909 13:49:05.263511 52398 iamrole.go:89] ListRoles: { } I0909 13:49:05.263546 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:05.263564 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:05.263588 52398 internetgateways.go:101] DescribeInternetGateways: { } I0909 13:49:05.263613 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:05.263633 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok autoscaling-group:nodes.minimal.example.com ok I0909 13:49:05.263925 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0909 13:49:05.263974 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0909 13:49:05.263969 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } iam-instance-profile:masters.minimal.example.com ok I0909 13:49:05.263995 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0909 13:49:05.264023 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0909 13:49:05.264022 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0909 13:49:05.264176 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:05.264005 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:05.264194 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.minimal.example.com. ok security-group:sg-1 ok I0909 13:49:05.264216 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok I0909 13:49:05.264270 52398 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0909 13:49:05.264308 52398 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I0909 13:49:05.264336 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:05.264365 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0909 13:49:05.264432 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:05.264441 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0909 13:49:05.264462 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0909 13:49:05.264491 52398 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0909 13:49:05.264544 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0909 13:49:05.264562 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0909 13:49:05.264623 52398 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0909 13:49:05.264639 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0909 13:49:05.264672 52398 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0909 13:49:05.264703 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0909 13:49:05.264741 52398 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0909 13:49:05.264754 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0909 13:49:05.264781 52398 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0909 13:49:05.264832 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0909 13:49:05.264901 52398 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0909 13:49:05.264967 52398 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0909 13:49:05.265212 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.35s) === RUN TestLifecycleMinimalOpenstack I0909 13:49:05.265367 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0909 13:49:05.270454 52398 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0909 13:49:05.456021 52398 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0909 13:49:05.456077 52398 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0909 13:49:05.465732 52398 executor.go:111] Tasks: 0 done / 90 total; 41 can run I0909 13:49:05.483468 52398 executor.go:111] Tasks: 41 done / 90 total; 41 can run I0909 13:49:05.534869 52398 executor.go:111] Tasks: 82 done / 90 total; 5 can run I0909 13:49:05.536938 52398 executor.go:111] Tasks: 87 done / 90 total; 3 can run 2023/09/09 13:49:05 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/09 13:49:05 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/09 13:49:05 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0909 13:49:05.538793 52398 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0909 13:49:05.538886 52398 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0909 13:49:05.538993 52398 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0909 13:49:06.547514 52398 executor.go:111] Tasks: 90 done / 90 total; 0 can run ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0909 13:49:06.703191 52398 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0909 13:49:06.703231 52398 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0909 13:49:06.714200 52398 executor.go:111] Tasks: 0 done / 90 total; 41 can run I0909 13:49:06.717987 52398 executor.go:111] Tasks: 41 done / 90 total; 41 can run I0909 13:49:06.724365 52398 executor.go:111] Tasks: 82 done / 90 total; 5 can run I0909 13:49:06.725634 52398 executor.go:111] Tasks: 87 done / 90 total; 3 can run 2023/09/09 13:49:06 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/09 13:49:06 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/09 13:49:06 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0909 13:49:06.727359 52398 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0909 13:49:06.727917 52398 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0909 13:49:06.728655 52398 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0909 13:49:06.729561 52398 delete_cluster.go:128] Looking for cloud resources to delete 2023/09/09 13:49:06 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) SecurityGroup:297d73d9-f20f-4a37-a6c3-d12dd4dea314 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Instance:fa738630-a73b-4e18-b434-a4e40bf2ceb4 ok Port:2b60a2b7-457b-4076-a530-6e535eb4f688 ok Network:1ea39e09-6fd1-49b0-adbb-7ceef11ad7f0 ok Subnet:2b966c0e-a32f-4e5d-abd3-37e7fc578afa ok SecurityGroup:1f6e5619-c7c7-41e5-9f7b-1ba50e1dfd9d ok Port:833bbc98-7049-4a9b-9c7e-636d7f26ec3a ok Port:b222ea8f-4d98-4537-817f-021066feff9f ok Instance:d8b33590-41d8-4064-89f1-c81b0464500f ok ServerGroup:59e24db8-5ca5-4f6a-8380-037f49627660 ok ServerGroup:fe7a97d8-21b4-49b0-8bc6-26e1e0e7133a ok SecurityGroup:733770df-d1dd-45ef-9fb3-60436cd4e26f ok Instance:644b8ac0-2aa0-4eba-8f8c-d644aa572e28 ok --- PASS: TestLifecycleMinimalOpenstack (2.56s) === RUN TestLifecycleMinimalGCE I0909 13:49:07.823220 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0909 13:49:07.825063 52398 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0909 13:49:08.059589 52398 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0909 13:49:08.059655 52398 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0909 13:49:08.059730 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0909 13:49:08.059750 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0909 13:49:08.059765 52398 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0909 13:49:08.059814 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0909 13:49:08.060715 52398 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0909 13:49:08.062815 52398 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-control-plane" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0909 13:49:08.062881 52398 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0909 13:49:08.067305 52398 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0909 13:49:08.068874 52398 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0909 13:49:08.069107 52398 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0909 13:49:08.069174 52398 dns.go:232] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0909 13:49:08.279736 52398 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0909 13:49:08.279816 52398 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0909 13:49:08.279901 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0909 13:49:08.279924 52398 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0909 13:49:08.279941 52398 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0909 13:49:08.279990 52398 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0909 13:49:08.280898 52398 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0909 13:49:08.282939 52398 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0909 13:49:08.287372 52398 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0909 13:49:08.289290 52398 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0909 13:49:08.289417 52398 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0909 13:49:08.290866 52398 delete_cluster.go:128] Looking for cloud resources to delete I0909 13:49:08.290908 52398 gce.go:99] Scanning zones: [us-test1-a] I0909 13:49:08.291260 52398 gce.go:1074] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0909 13:49:08.291281 52398 gce.go:1074] backend services not found, assuming none exist in project: "testproject" region: "us-test1" FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok Disk:a-etcd-main-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok Disk:a-etcd-events-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1694267348 ok InstanceTemplate:nodes-minimal-gce-example-com-1694267348 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.47s) === RUN TestLifecycleFloatingIPOpenstack I0909 13:49:08.292230 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0909 13:49:08.297398 52398 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0909 13:49:08.470531 52398 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0909 13:49:08.470566 52398 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0909 13:49:08.482423 52398 executor.go:111] Tasks: 0 done / 92 total; 42 can run I0909 13:49:08.485767 52398 executor.go:111] Tasks: 42 done / 92 total; 41 can run I0909 13:49:08.490950 52398 executor.go:111] Tasks: 83 done / 92 total; 6 can run I0909 13:49:08.492846 52398 executor.go:111] Tasks: 89 done / 92 total; 3 can run 2023/09/09 13:49:08 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/09 13:49:08 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/09 13:49:08 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0909 13:49:08.494936 52398 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0909 13:49:08.495401 52398 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0909 13:49:08.495436 52398 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0909 13:49:09.502380 52398 executor.go:111] Tasks: 92 done / 92 total; 0 can run ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0909 13:49:09.713005 52398 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0909 13:49:09.713052 52398 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0909 13:49:09.732520 52398 executor.go:111] Tasks: 0 done / 92 total; 42 can run I0909 13:49:09.735569 52398 executor.go:111] Tasks: 42 done / 92 total; 41 can run I0909 13:49:09.740481 52398 executor.go:111] Tasks: 83 done / 92 total; 6 can run I0909 13:49:09.741700 52398 executor.go:111] Tasks: 89 done / 92 total; 3 can run 2023/09/09 13:49:09 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/09 13:49:09 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/09 13:49:09 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0909 13:49:09.743864 52398 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0909 13:49:09.744353 52398 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0909 13:49:09.745308 52398 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0909 13:49:09.746279 52398 delete_cluster.go:128] Looking for cloud resources to delete 2023/09/09 13:49:09 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) Port:9e82c718-f775-4207-aad8-34da6da4c19f ok Port:f8a19e72-e5f8-47fa-868f-c51ff150d9e4 ok SecurityGroup:7034d56b-976f-4005-80fc-cdcbedbc10cc ok ServerGroup:767dfd4e-593a-4138-9d28-b51906b88587 ok Router:3d209fc9-957e-4ed7-a7cc-3baa5c8e5ed7 ok SecurityGroup:8a740bf1-6f28-41f6-b911-3e43c308a807 ok SecurityGroup:89474664-7680-4e60-8f58-24dbfe60d190 ok Instance:c027d6fb-9d23-4601-b331-f4b201af508e ok Instance:b5ac9b43-2e4c-403f-8438-8529575d5cdc ok Network:46e6955c-4767-4785-a594-9027c588114c ok Port:698899a6-afac-4057-a79b-0180e9f07ae4 ok Instance:973ff53d-defe-46cd-af58-f1c0dd035f24 ok ServerGroup:4ea08c5d-dd7a-4325-85c3-2907b2bd48cd ok Port:a22eb112-1989-451f-ad93-790d7a7316b0 ok Subnet:d42475d6-7c68-4ba2-b723-fb7255798275 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Router-IF:d42475d6-7c68-4ba2-b723-fb7255798275 ok --- PASS: TestLifecycleFloatingIPOpenstack (2.56s) === RUN TestLifecyclePrivateCalico I0909 13:49:10.856223 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:10.856405 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:10.856444 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:10.856491 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:10.856531 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:10.856580 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:10.856617 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:10.856644 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:10.856680 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:10.856705 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:10.856753 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0909 13:49:10.883959 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:49:10.887017 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:10.887094 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:10.887150 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:10.961666 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:11.265982 52398 iamrole.go:89] ListRoles: { } I0909 13:49:11.267263 52398 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0909 13:49:11.267478 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:11.267574 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0909 13:49:11.267611 52398 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.267667 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0909 13:49:11.267773 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0909 13:49:11.267816 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0909 13:49:11.267770 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.267871 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:11.267951 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.269903 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] }] } I0909 13:49:11.270037 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0909 13:49:11.270159 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0909 13:49:11.270146 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:11.270368 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:11.270478 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.270750 52398 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.270887 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:11.270869 52398 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.270936 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:11.271155 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:11.271289 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0909 13:49:11.271342 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0909 13:49:11.271372 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:11.271414 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:11.272539 52398 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0909 13:49:11.272645 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0909 13:49:11.273200 52398 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0909 13:49:11.273096 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0909 13:49:11.273547 52398 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0909 13:49:11.273215 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatecalico-exa-hocohm"] } I0909 13:49:11.273553 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0909 13:49:11.273656 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0909 13:49:11.273679 52398 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-privatecalico-exa-hocohm", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-privatecalico-exa-hocohm" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0909 13:49:11.273777 52398 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } I0909 13:49:11.273773 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:11.273728 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0909 13:49:11.273853 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:11.273896 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:11.273923 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.273925 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0909 13:49:11.273983 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0909 13:49:11.274027 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0909 13:49:11.274099 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0909 13:49:11.274135 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0909 13:49:11.274139 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0909 13:49:11.274256 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:11.274314 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.274372 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.274435 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:11.274481 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:11.274525 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0909 13:49:11.274586 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.274546 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0909 13:49:11.274642 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.274697 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:11.274751 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:11.274698 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0909 13:49:11.274780 52398 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] } I0909 13:49:11.274828 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:11.274855 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:11.274891 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:11.274919 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:11.274942 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0909 13:49:11.275012 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0909 13:49:11.275051 52398 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0909 13:49:11.275114 52398 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0909 13:49:11.275158 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:11.275258 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0909 13:49:11.275312 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0909 13:49:11.275371 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:11.275419 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:11.275461 52398 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.275243 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0909 13:49:11.275549 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:11.275598 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:11.275646 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:11.275689 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0909 13:49:11.275736 52398 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0909 13:49:11.275870 52398 executor.go:111] Tasks: 76 done / 128 total; 37 can run W0909 13:49:11.275974 52398 api.go:53] PageSize not implemented I0909 13:49:11.275949 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:11.276028 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:11.276076 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.276090 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0909 13:49:11.276131 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0909 13:49:11.276138 52398 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } W0909 13:49:11.276187 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:11.276161 52398 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatecalico.example.com" }] } I0909 13:49:11.276200 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.276221 52398 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0909 13:49:11.276227 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0909 13:49:11.276256 52398 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0909 13:49:11.276264 52398 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-privatecalico-exa-hocohm", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }], Type: "network" } I0909 13:49:11.276293 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0909 13:49:11.276399 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I0909 13:49:11.276505 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.276629 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.276710 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.276830 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0909 13:49:11.276948 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" }] }] } I0909 13:49:11.277098 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0909 13:49:11.277227 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:11.277285 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:11.277329 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0909 13:49:11.277408 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.277496 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0909 13:49:11.277528 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:11.277580 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:11.277603 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.277687 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0909 13:49:11.277711 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0909 13:49:11.277695 52398 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1", Port: 22, Protocol: "TCP" } W0909 13:49:11.277790 52398 api.go:53] PageSize not implemented I0909 13:49:11.277812 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.277733 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" }] }] } I0909 13:49:11.277854 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.277957 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.278043 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0909 13:49:11.278077 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.278162 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.278243 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0909 13:49:11.278301 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.278380 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.278443 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.278507 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.278593 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.278673 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0909 13:49:11.278695 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0909 13:49:11.278715 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0909 13:49:11.278797 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0909 13:49:11.278874 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.278952 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.279046 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.279139 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0909 13:49:11.279193 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" }] }] } I0909 13:49:11.279292 52398 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0909 13:49:11.279337 52398 api.go:53] PageSize not implemented I0909 13:49:11.279386 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.279393 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:11.279409 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } W0909 13:49:11.279480 52398 api.go:53] PageSize not implemented I0909 13:49:11.279465 52398 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:11.279509 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.279555 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:11.279603 52398 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0909 13:49:11.279713 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0909 13:49:11.279758 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:11.279800 52398 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0909 13:49:11.282329 52398 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0909 13:49:11.282459 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.282536 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.282603 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.282684 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.282759 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:11.282888 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0909 13:49:11.284059 52398 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0909 13:49:11.284208 52398 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0909 13:49:11.284282 52398 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0909 13:49:11.284312 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:11.284334 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:11.284372 52398 dns.go:232] Pre-creating DNS records I0909 13:49:11.284388 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:11.288084 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:49:11.290061 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.290118 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.290154 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.360891 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:11.672541 52398 iamrole.go:89] ListRoles: { } I0909 13:49:11.673827 52398 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0909 13:49:11.674056 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:11.674293 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:11.674426 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.674552 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:11.674582 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:11.674611 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.674710 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] }] } I0909 13:49:11.674798 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.674878 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:11.676109 52398 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0909 13:49:11.676215 52398 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0909 13:49:11.676212 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatecalico-exa-hocohm"] } I0909 13:49:11.676294 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4"] } I0909 13:49:11.676315 52398 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0909 13:49:11.676332 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } I0909 13:49:11.676372 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0909 13:49:11.676275 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0909 13:49:11.676441 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.676566 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:11.676598 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0909 13:49:11.676708 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0909 13:49:11.676747 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0909 13:49:11.676843 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:11.676875 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.676968 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.677052 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0909 13:49:11.677150 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:11.677209 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.677302 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:11.677360 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:11.677412 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:11.677458 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.677532 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:11.677578 52398 executor.go:111] Tasks: 76 done / 128 total; 37 can run W0909 13:49:11.677712 52398 api.go:53] PageSize not implemented I0909 13:49:11.677710 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:11.677736 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.677783 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:11.677833 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:11.677810 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.677813 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:11.677846 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0909 13:49:11.677871 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0909 13:49:11.677875 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0909 13:49:11.677906 52398 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0909 13:49:11.677932 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } I0909 13:49:11.677878 52398 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0909 13:49:11.677956 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0909 13:49:11.677980 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4"] } I0909 13:49:11.678013 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } I0909 13:49:11.678007 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0909 13:49:11.678082 52398 api.go:53] PageSize not implemented I0909 13:49:11.678099 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.678087 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.678146 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.678209 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:11.678446 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:11.678577 52398 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0909 13:49:11.678638 52398 api.go:53] PageSize not implemented I0909 13:49:11.678622 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:11.678638 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:11.678685 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0909 13:49:11.678708 52398 dnsname.go:118] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I0909 13:49:11.678679 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.678822 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } W0909 13:49:11.678883 52398 api.go:53] PageSize not implemented I0909 13:49:11.678925 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.684599 52398 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0909 13:49:11.684694 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.684801 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:11.684959 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.685165 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.685289 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.687292 52398 executor.go:111] Tasks: 124 done / 128 total; 2 can run W0909 13:49:11.687383 52398 api.go:53] PageSize not implemented I0909 13:49:11.687403 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.687525 52398 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0909 13:49:11.687618 52398 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0909 13:49:11.690662 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0909 13:49:11.690728 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:11.690765 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0909 13:49:11.690796 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0909 13:49:11.690830 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:11.690858 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0909 13:49:11.690891 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0909 13:49:11.690921 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:11.690958 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0909 13:49:11.690993 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0909 13:49:11.691029 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0909 13:49:11.691067 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0909 13:49:11.691108 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0909 13:49:11.691146 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:11.691180 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:11.691213 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:11.691249 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0909 13:49:11.691283 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0909 13:49:11.691319 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:11.691354 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:11.691388 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:11.691424 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0909 13:49:11.691458 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:11.691513 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:11.691578 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:11.691645 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:11.691699 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0909 13:49:11.691766 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:11.691819 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:11.691858 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:11.691892 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0909 13:49:11.693411 52398 delete_cluster.go:128] Looking for cloud resources to delete W0909 13:49:11.693475 52398 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0909 13:49:11.693495 52398 keypairs.go:95] DescribeKeyPairs: { } I0909 13:49:11.693534 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.693591 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0909 13:49:11.693655 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.693745 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.693792 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.693852 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.693897 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.693940 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0909 13:49:11.694002 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.694057 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:11.694086 52398 natgateway.go:117] DescribeNatGateways: { } I0909 13:49:11.694123 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0909 13:49:11.694165 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0909 13:49:11.694216 52398 api.go:53] PageSize not implemented I0909 13:49:11.694233 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0909 13:49:11.694276 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:11.694301 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:11.694317 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0909 13:49:11.694352 52398 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0909 13:49:11.694376 52398 targetgroups.go:35] PageSize not implemented I0909 13:49:11.694392 52398 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I0909 13:49:11.694447 52398 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0909 13:49:11.694487 52398 iamrole.go:89] ListRoles: { } I0909 13:49:11.694534 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:11.694561 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:11.694600 52398 internetgateways.go:101] DescribeInternetGateways: { } I0909 13:49:11.694643 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:11.694680 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0909 13:49:11.694714 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0909 13:49:11.694750 52398 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:nodes.privatecalico.example.com ok I0909 13:49:11.695338 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0909 13:49:11.695336 52398 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok keypair:key-1 ok I0909 13:49:11.695407 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } autoscaling-group:bastion.privatecalico.example.com ok iam-instance-profile:masters.privatecalico.example.com ok I0909 13:49:11.695434 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0909 13:49:11.695437 52398 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I0909 13:49:11.695482 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0909 13:49:11.695484 52398 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4 ok I0909 13:49:11.695436 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0909 13:49:11.695540 52398 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1 ok I0909 13:49:11.695564 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:11.695649 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0909 13:49:11.695538 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatecalico.example.com ok I0909 13:49:11.695723 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0909 13:49:11.695625 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0909 13:49:11.695781 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:11.695798 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0909 13:49:11.695822 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0909 13:49:11.695856 52398 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0909 13:49:11.695897 52398 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-3 ok autoscaling-config:lt-1 ok I0909 13:49:11.695932 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:11.695961 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0909 13:49:11.696031 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:11.696041 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0909 13:49:11.696060 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0909 13:49:11.696091 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 } ] } W0909 13:49:11.696192 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:11.696205 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-2 ok I0909 13:49:11.696232 52398 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0909 13:49:11.696305 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0909 13:49:11.696337 52398 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I0909 13:49:11.696314 52398 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0909 13:49:11.696368 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:11.696354 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0909 13:49:11.696438 52398 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0909 13:49:11.696455 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0909 13:49:11.696491 52398 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0909 13:49:11.696407 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 } ] } I0909 13:49:11.696517 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } W0909 13:49:11.696537 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:11.696550 52398 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0909 13:49:11.696552 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } I0909 13:49:11.696558 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } security-group:sg-3 ok I0909 13:49:11.696577 52398 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0909 13:49:11.696597 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0909 13:49:11.696614 52398 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0909 13:49:11.696590 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0909 13:49:11.696656 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0909 13:49:11.696719 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0909 13:49:11.696753 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0909 13:49:11.696793 52398 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0909 13:49:11.696835 52398 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0909 13:49:11.697075 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.84s) === RUN TestLifecyclePrivateKopeio I0909 13:49:11.697232 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:11.697325 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:11.697348 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:11.697374 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:11.697396 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:11.697428 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:11.697455 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:11.697473 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:11.697496 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:11.697513 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:11.697543 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0909 13:49:11.714750 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:11.716650 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.716707 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.716743 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.787265 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:11.858454 52398 iamrole.go:89] ListRoles: { } I0909 13:49:11.859780 52398 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0909 13:49:11.860012 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0909 13:49:11.860049 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.860131 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0909 13:49:11.860250 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.860388 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0909 13:49:11.860454 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:11.860479 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0909 13:49:11.860486 52398 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.860634 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0909 13:49:11.860702 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0909 13:49:11.860526 52398 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.861522 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:11.861585 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:11.861798 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.862205 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:11.864847 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.864971 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:11.865083 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0909 13:49:11.865132 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0909 13:49:11.865164 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:11.865208 52398 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.865838 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:11.865998 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:11.866048 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:11.866138 52398 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0909 13:49:11.866230 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.866307 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0909 13:49:11.866324 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatekopeio-exa-d8ef8e"] } I0909 13:49:11.866388 52398 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0909 13:49:11.866347 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0909 13:49:11.866419 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0909 13:49:11.866436 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I0909 13:49:11.866508 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0909 13:49:11.866544 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:11.866488 52398 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-privatekopeio-exa-d8ef8e", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-privatekopeio-exa-d8ef8e" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0909 13:49:11.866593 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:11.866621 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:11.866617 52398 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } I0909 13:49:11.866689 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0909 13:49:11.866757 52398 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0909 13:49:11.866753 52398 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0909 13:49:11.866792 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0909 13:49:11.866829 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0909 13:49:11.866937 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:11.867011 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:11.867026 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0909 13:49:11.867041 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:11.867080 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.867219 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-1" } I0909 13:49:11.867206 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.867352 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0909 13:49:11.867417 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:11.867512 52398 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0909 13:49:11.867478 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0909 13:49:11.867553 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.867737 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0909 13:49:11.867746 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.867943 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.867946 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0909 13:49:11.868050 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0909 13:49:11.868114 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.868180 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0909 13:49:11.868167 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0909 13:49:11.868254 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0909 13:49:11.868246 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0909 13:49:11.868296 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0909 13:49:11.868436 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0909 13:49:11.868489 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0909 13:49:11.868333 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0909 13:49:11.868531 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0909 13:49:11.868639 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:11.868734 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:11.868843 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:11.868903 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:11.868932 52398 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" } ] } I0909 13:49:11.869006 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:11.869070 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:11.869103 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:11.869126 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:11.869193 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0909 13:49:11.869223 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0909 13:49:11.869246 52398 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.869310 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:11.869349 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0909 13:49:11.869390 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:11.869428 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:11.869454 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:11.869490 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0909 13:49:11.869529 52398 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I0909 13:49:11.869576 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:11.869611 52398 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" } ] } I0909 13:49:11.869697 52398 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0909 13:49:11.869816 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0909 13:49:11.869892 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0909 13:49:11.869959 52398 api.go:53] PageSize not implemented I0909 13:49:11.869975 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:11.869926 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0909 13:49:11.870026 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } W0909 13:49:11.870058 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:11.870101 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0909 13:49:11.870134 52398 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-privatekopeio-exa-d8ef8e", SubnetMappings: [{ SubnetId: "subnet-2" },{ SubnetId: "subnet-3" }], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" }], Type: "network" } I0909 13:49:11.870116 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.870183 52398 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatekopeio.example.com" }] } I0909 13:49:11.870251 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0909 13:49:11.870285 52398 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0909 13:49:11.870277 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.870341 52398 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0909 13:49:11.870378 52398 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0909 13:49:11.870435 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:11.870490 52398 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0909 13:49:11.870550 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0909 13:49:11.870592 52398 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0909 13:49:11.870708 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 4 }] } I0909 13:49:11.870819 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.870957 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0909 13:49:11.871167 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.871304 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:11.871377 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:11.871452 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0909 13:49:11.871573 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.871725 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:11.871797 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.871904 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.871988 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0909 13:49:11.872056 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:11.872127 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:11.872205 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0909 13:49:11.872263 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I0909 13:49:11.872307 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" }] }] } I0909 13:49:11.872471 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0909 13:49:11.872515 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0909 13:49:11.872557 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0909 13:49:11.872599 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-4" } I0909 13:49:11.872635 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0909 13:49:11.872774 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.872908 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.873054 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.873215 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.873358 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" }] }] } I0909 13:49:11.873503 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.873644 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" }] }] } I0909 13:49:11.873783 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0909 13:49:11.873974 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.874118 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-172.20.8.0/22-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.874260 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0909 13:49:11.874400 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0909 13:49:11.874579 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0909 13:49:11.874847 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0909 13:49:11.875115 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0909 13:49:11.875221 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-1" } I0909 13:49:11.875409 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.privatekopeio.example.com" }] }] } I0909 13:49:11.875577 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:11.875624 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0909 13:49:11.875678 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:11.875716 52398 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0909 13:49:11.875763 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0909 13:49:11.875805 52398 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0909 13:49:11.875844 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:11.875883 52398 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0909 13:49:11.875929 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0909 13:49:11.876025 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:11.876077 52398 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1", Port: 22, Protocol: "TCP" } W0909 13:49:11.876166 52398 api.go:53] PageSize not implemented I0909 13:49:11.876185 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0909 13:49:11.876259 52398 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0909 13:49:11.876300 52398 api.go:53] PageSize not implemented I0909 13:49:11.876333 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } W0909 13:49:11.876387 52398 api.go:53] PageSize not implemented I0909 13:49:11.876358 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:11.876392 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:11.876433 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0909 13:49:11.876450 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:11.876513 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-3" } I0909 13:49:11.876502 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:11.876576 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-1" } I0909 13:49:11.879275 52398 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0909 13:49:11.879424 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.879512 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.879592 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.879693 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.881914 52398 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0909 13:49:11.882118 52398 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0909 13:49:11.882230 52398 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0909 13:49:11.882274 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:11.882306 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:11.882365 52398 dns.go:232] Pre-creating DNS records I0909 13:49:11.882388 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:11.889324 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:11.892519 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.892609 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.892674 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:11.963561 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.033216 52398 iamrole.go:89] ListRoles: { } I0909 13:49:12.034503 52398 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0909 13:49:12.034639 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.034730 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:12.034936 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.035109 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:12.035218 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.036078 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.036226 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.036266 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.036343 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.037990 52398 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0909 13:49:12.038319 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.038606 52398 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0909 13:49:12.038531 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.038642 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatekopeio-exa-d8ef8e"] } I0909 13:49:12.038881 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0909 13:49:12.038878 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4"] } I0909 13:49:12.039052 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } I0909 13:49:12.039103 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:12.039144 52398 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0909 13:49:12.039185 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.039357 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0909 13:49:12.039781 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:12.039832 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:12.039872 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.039939 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.040026 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0909 13:49:12.040049 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0909 13:49:12.040588 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:12.040624 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.040713 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.040739 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.040818 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0909 13:49:12.040902 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0909 13:49:12.040938 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.041016 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0909 13:49:12.041109 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:12.041162 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.041271 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.041360 52398 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0909 13:49:12.041513 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.041535 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:12.041615 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:12.041601 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0909 13:49:12.041636 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0909 13:49:12.041685 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0909 13:49:12.041666 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0909 13:49:12.041595 52398 api.go:53] PageSize not implemented I0909 13:49:12.041712 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } I0909 13:49:12.041784 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.041752 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0909 13:49:12.041824 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4"] } I0909 13:49:12.041860 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0909 13:49:12.041849 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.041909 52398 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0909 13:49:12.041865 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } I0909 13:49:12.041948 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0909 13:49:12.042010 52398 api.go:53] PageSize not implemented I0909 13:49:12.042026 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0909 13:49:12.042028 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0909 13:49:12.042116 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:12.042215 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.042276 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.042521 52398 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0909 13:49:12.042560 52398 api.go:53] PageSize not implemented I0909 13:49:12.042588 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0909 13:49:12.042596 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:12.042671 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:12.042687 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.042732 52398 dnsname.go:118] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0909 13:49:12.042748 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } W0909 13:49:12.042788 52398 api.go:53] PageSize not implemented I0909 13:49:12.042811 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0909 13:49:12.045510 52398 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0909 13:49:12.045582 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.045703 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.045750 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.045911 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.047221 52398 executor.go:111] Tasks: 124 done / 128 total; 2 can run W0909 13:49:12.047308 52398 api.go:53] PageSize not implemented I0909 13:49:12.047330 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0909 13:49:12.047500 52398 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0909 13:49:12.047624 52398 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0909 13:49:12.051502 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0909 13:49:12.051569 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:12.051610 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0909 13:49:12.051650 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:12.051687 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0909 13:49:12.051725 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0909 13:49:12.051762 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:12.051803 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0909 13:49:12.051843 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0909 13:49:12.051888 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0909 13:49:12.051932 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0909 13:49:12.051978 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:12.052017 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:12.052056 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:12.052097 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0909 13:49:12.052133 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0909 13:49:12.052175 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0909 13:49:12.052216 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:12.052256 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:12.052298 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:12.052339 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0909 13:49:12.052379 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.052422 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:12.052460 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:12.052502 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0909 13:49:12.052543 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0909 13:49:12.052584 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:12.052620 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0909 13:49:12.052653 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:12.052698 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:12.052743 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.052783 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0909 13:49:12.054605 52398 delete_cluster.go:128] Looking for cloud resources to delete W0909 13:49:12.054675 52398 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0909 13:49:12.054696 52398 keypairs.go:95] DescribeKeyPairs: { } I0909 13:49:12.054750 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.054813 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0909 13:49:12.054885 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.054951 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.055002 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.055057 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.055104 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.055159 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0909 13:49:12.055228 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.055299 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:12.055335 52398 natgateway.go:117] DescribeNatGateways: { } I0909 13:49:12.055377 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0909 13:49:12.055422 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0909 13:49:12.055476 52398 api.go:53] PageSize not implemented I0909 13:49:12.055494 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0909 13:49:12.055537 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:12.055567 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:12.055586 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0909 13:49:12.055626 52398 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0909 13:49:12.055652 52398 targetgroups.go:35] PageSize not implemented I0909 13:49:12.055670 52398 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I0909 13:49:12.055748 52398 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0909 13:49:12.055790 52398 iamrole.go:89] ListRoles: { } I0909 13:49:12.055839 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.055866 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:12.055908 52398 internetgateways.go:101] DescribeInternetGateways: { } I0909 13:49:12.055954 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:12.056002 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2","rtb-3"] } I0909 13:49:12.056045 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0909 13:49:12.056077 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0909 13:49:12.056778 52398 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok autoscaling-config:lt-3 ok autoscaling-group:nodes.privatekopeio.example.com ok autoscaling-group:bastion.privatekopeio.example.com ok autoscaling-config:lt-2 ok I0909 13:49:12.056853 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0909 13:49:12.056865 52398 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1 ok I0909 13:49:12.056903 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0909 13:49:12.056927 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } keypair:key-1 ok I0909 13:49:12.056914 52398 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I0909 13:49:12.056950 52398 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } iam-instance-profile:nodes.privatekopeio.example.com ok target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4 ok I0909 13:49:12.056965 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0909 13:49:12.057014 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I0909 13:49:12.056954 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:12.057050 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0909 13:49:12.057089 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I0909 13:49:12.056941 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatekopeio.example.com ok I0909 13:49:12.057067 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W0909 13:49:12.057268 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:12.057284 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0909 13:49:12.057309 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0909 13:49:12.057345 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0909 13:49:12.057377 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0909 13:49:12.057464 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:12.057472 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-1 ok I0909 13:49:12.057507 52398 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0909 13:49:12.057545 52398 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0909 13:49:12.057574 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:12.057602 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 } ] } W0909 13:49:12.057715 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:12.057723 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0909 13:49:12.057795 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } I0909 13:49:12.057801 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } subnet:subnet-3 ok I0909 13:49:12.057850 52398 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0909 13:49:12.057848 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:12.057866 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0909 13:49:12.057898 52398 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0909 13:49:12.057928 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0909 13:49:12.057891 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0909 13:49:12.057991 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:12.058007 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0909 13:49:12.057973 52398 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0909 13:49:12.058033 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0909 13:49:12.058061 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0909 13:49:12.058059 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0909 13:49:12.058094 52398 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0909 13:49:12.058089 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0909 13:49:12.058112 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0909 13:49:12.058135 52398 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0909 13:49:12.058143 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0909 13:49:12.058161 52398 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0909 13:49:12.058203 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0909 13:49:12.058234 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0909 13:49:12.058264 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0909 13:49:12.058305 52398 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0909 13:49:12.058347 52398 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0909 13:49:12.058529 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.36s) === RUN TestLifecycleIPv6 I0909 13:49:12.058654 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:12.058743 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:12.058774 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:12.058801 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:12.058824 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:12.058855 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:12.058879 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:12.058898 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:12.058922 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:12.058939 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:12.058968 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0909 13:49:12.075610 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:49:12.077724 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.077791 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.148614 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.284680 52398 iamrole.go:89] ListRoles: { } I0909 13:49:12.285925 52398 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0909 13:49:12.286153 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.286368 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:12.286367 52398 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.286511 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:12.286546 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0909 13:49:12.287305 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0909 13:49:12.287404 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0909 13:49:12.287548 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0909 13:49:12.287653 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.287884 52398 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.288064 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:12.288228 52398 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.288362 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.288513 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:12.288624 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:12.288667 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.288801 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:12.288911 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:12.288966 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0909 13:49:12.289002 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0909 13:49:12.289049 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.289092 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.291013 52398 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0909 13:49:12.291573 52398 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0909 13:49:12.291603 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0909 13:49:12.291559 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0909 13:49:12.291652 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0909 13:49:12.291662 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0909 13:49:12.291683 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.291776 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0909 13:49:12.291836 52398 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0909 13:49:12.291885 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0909 13:49:12.291969 52398 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0909 13:49:12.292098 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.292063 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0909 13:49:12.292249 52398 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0909 13:49:12.292299 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.292390 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0909 13:49:12.292482 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0909 13:49:12.292471 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.292588 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0909 13:49:12.292651 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0909 13:49:12.292724 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0909 13:49:12.292782 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:12.292871 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.292930 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.292984 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.293027 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:12.293076 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:12.293124 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:12.293178 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:12.293209 52398 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I0909 13:49:12.293251 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:12.293289 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:12.293312 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:12.293336 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:12.293364 52398 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.293408 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:12.293435 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0909 13:49:12.293505 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0909 13:49:12.293549 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:12.293307 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0909 13:49:12.293629 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:12.293671 52398 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0909 13:49:12.293724 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0909 13:49:12.293772 52398 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0909 13:49:12.293883 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0909 13:49:12.293933 52398 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0909 13:49:12.294050 52398 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0909 13:49:12.294122 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.294193 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.294286 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", Ipv6CidrBlock: "2001:db8:0:112::/64", Ipv6Native: true, TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0909 13:49:12.294349 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.294361 52398 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.294447 52398 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0909 13:49:12.294552 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0909 13:49:12.294741 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0909 13:49:12.294903 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0909 13:49:12.295038 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.295195 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:113::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "dualstack-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "DualStack" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I0909 13:49:12.295293 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.295486 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:12.295541 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:12.295586 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.295667 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.295765 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.295855 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.295845 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.0.0/22", Ipv6CidrBlock: "2001:db8:0:115::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0909 13:49:12.295952 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:12.295937 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.4.0/22", Ipv6CidrBlock: "2001:db8:0:116::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0909 13:49:12.295998 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", Ipv6CidrBlock: "2001:db8:0:111::/64", Ipv6Native: true, TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-1" } I0909 13:49:12.296021 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.296093 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:12.296158 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["dualstack-us-test-1b.minimal-ipv6.example.com"] }] } I0909 13:49:12.296303 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", Ipv6CidrBlock: "2001:db8:0:114::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "DualStack" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "dualstack-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0909 13:49:12.296253 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.296417 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I0909 13:49:12.296562 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0909 13:49:12.296711 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0909 13:49:12.296810 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", EgressOnlyInternetGatewayId: "eigw-2", RouteTableId: "rtb-1" } I0909 13:49:12.296864 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.297008 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.297144 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.297296 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.297437 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.297589 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0909 13:49:12.297638 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:12.297686 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" }] }] } I0909 13:49:12.297820 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.297967 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I0909 13:49:12.298011 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.298146 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", EgressOnlyInternetGatewayId: "eigw-2", RouteTableId: "rtb-2" } I0909 13:49:12.298188 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.298327 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I0909 13:49:12.298459 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I0909 13:49:12.298523 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.298652 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.298792 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-5"] }] } I0909 13:49:12.298841 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-5"] } I0909 13:49:12.298880 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-6"] }] } I0909 13:49:12.298923 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-6"] } I0909 13:49:12.298960 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0909 13:49:12.299011 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0909 13:49:12.299049 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.299093 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.299166 52398 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0909 13:49:12.299241 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.299241 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:12.299300 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:12.299310 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:12.299413 52398 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-3" },{ SubnetId: "subnet-4" }], Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api.minimal-ipv6.example.com" }], Type: "network" } I0909 13:49:12.299464 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.299638 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-5"] }] } I0909 13:49:12.299685 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-5" } I0909 13:49:12.299724 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.299798 52398 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-3"] }] } I0909 13:49:12.299871 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.299914 52398 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-2", SubnetId: "subnet-3", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0909 13:49:12.299960 52398 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-4"] }] } I0909 13:49:12.300137 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0909 13:49:12.300233 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0909 13:49:12.300115 52398 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-4", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0909 13:49:12.300289 52398 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0909 13:49:12.300355 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-2"] }] } I0909 13:49:12.300393 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0909 13:49:12.300428 52398 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-2" }] } I0909 13:49:12.300483 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0909 13:49:12.300573 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0909 13:49:12.300628 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:12.300806 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.300967 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0909 13:49:12.301013 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0909 13:49:12.301059 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:12.301101 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.301168 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0909 13:49:12.301209 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-3" } I0909 13:49:12.301245 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-6"] }] } I0909 13:49:12.301277 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-6" } I0909 13:49:12.301312 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I0909 13:49:12.301356 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0909 13:49:12.301410 52398 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0909 13:49:12.301475 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0909 13:49:12.301496 52398 loadbalancers.go:170] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } W0909 13:49:12.301563 52398 api.go:53] PageSize not implemented I0909 13:49:12.301621 52398 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0909 13:49:12.301668 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:12.301718 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:12.301698 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.301734 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0909 13:49:12.301686 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:12.301764 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.301886 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-2", RouteTableId: "rtb-1" } I0909 13:49:12.301879 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:12.301938 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.301972 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:12.301993 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:12.302045 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:12.302041 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0909 13:49:12.302061 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0909 13:49:12.302084 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.302081 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:12.302191 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0909 13:49:12.302231 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-2", RouteTableId: "rtb-1" } I0909 13:49:12.305079 52398 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0909 13:49:12.305203 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.305284 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.305417 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.305525 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.307100 52398 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0909 13:49:12.307332 52398 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0909 13:49:12.307420 52398 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0909 13:49:12.307454 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.307480 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:12.307523 52398 dns.go:232] Pre-creating DNS records I0909 13:49:12.307539 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:12.313303 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0909 13:49:12.316922 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.317017 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.388153 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.538820 52398 iamrole.go:89] ListRoles: { } I0909 13:49:12.540775 52398 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0909 13:49:12.540998 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.541135 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.541173 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.541230 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.541272 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.541409 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:12.541526 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.541654 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.541219 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:12.545747 52398 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0909 13:49:12.545908 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.minimal-ipv6.example.com"] }] } I0909 13:49:12.546082 52398 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0909 13:49:12.546069 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0909 13:49:12.546130 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0909 13:49:12.546175 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0909 13:49:12.546260 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0909 13:49:12.546364 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0909 13:49:12.546319 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0909 13:49:12.546938 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:12.547007 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0909 13:49:12.547136 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:12.547201 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.547595 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.547699 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.547800 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.547872 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.547932 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.548020 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:12.548599 52398 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0909 13:49:12.548693 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.548825 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.548926 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } I0909 13:49:12.548971 52398 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0909 13:49:12.549015 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0909 13:49:12.549097 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] }] } I0909 13:49:12.549254 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-5"] } I0909 13:49:12.549475 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.549657 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:12.549765 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.549898 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:12.549948 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.550072 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1b.minimal-ipv6.example.com"] }] } I0909 13:49:12.550227 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0909 13:49:12.550275 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.550384 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.550525 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:12.550606 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-6"] } I0909 13:49:12.550647 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.550732 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:12.550774 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.550816 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0909 13:49:12.550857 52398 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-2"] } I0909 13:49:12.550896 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-2"] }] } I0909 13:49:12.550994 52398 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0909 13:49:12.551070 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.551139 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0909 13:49:12.551229 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.551331 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.551440 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:12.551502 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:12.551522 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0909 13:49:12.551565 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0909 13:49:12.551602 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0909 13:49:12.551642 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0909 13:49:12.551680 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } W0909 13:49:12.551770 52398 api.go:53] PageSize not implemented I0909 13:49:12.551818 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.551915 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.552017 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:12.552112 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.552175 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } I0909 13:49:12.552257 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0909 13:49:12.552352 52398 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0909 13:49:12.552387 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:12.552434 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:12.552448 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0909 13:49:12.553908 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:12.554001 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:12.553961 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.554022 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0909 13:49:12.554042 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:12.554083 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.554123 52398 dnsname.go:118] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0909 13:49:12.554146 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0909 13:49:12.554161 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0909 13:49:12.554156 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.554184 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:12.554240 52398 dnsname.go:118] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0909 13:49:12.554254 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0909 13:49:12.554268 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0909 13:49:12.554262 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:12.557976 52398 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0909 13:49:12.558121 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.558294 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.558368 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.558498 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.561017 52398 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0909 13:49:12.561209 52398 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0909 13:49:12.561352 52398 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0909 13:49:12.563339 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0909 13:49:12.563388 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:12.563414 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0909 13:49:12.563436 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-2"] }] } I0909 13:49:12.563458 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0909 13:49:12.563479 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:12.563499 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-2"] }] } I0909 13:49:12.563519 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0909 13:49:12.563538 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0909 13:49:12.563557 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:12.563581 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0909 13:49:12.563602 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0909 13:49:12.563624 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0909 13:49:12.563650 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0909 13:49:12.563690 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-2"] }] } I0909 13:49:12.563713 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:12.563734 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:12.563752 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:12.563773 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0909 13:49:12.563792 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0909 13:49:12.563814 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0909 13:49:12.563848 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:12.563877 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:12.563899 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:12.563920 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.563943 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:12.563965 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:12.563987 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0909 13:49:12.564009 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0909 13:49:12.564030 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-5"] }] } I0909 13:49:12.564051 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-6"] }] } I0909 13:49:12.564072 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:12.564090 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0909 13:49:12.564110 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:12.564133 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:12.564153 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.564177 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0909 13:49:12.565705 52398 delete_cluster.go:128] Looking for cloud resources to delete W0909 13:49:12.565816 52398 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0909 13:49:12.565838 52398 keypairs.go:95] DescribeKeyPairs: { } I0909 13:49:12.565874 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.565924 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0909 13:49:12.565962 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.566010 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.566043 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.566086 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.566118 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.566158 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0909 13:49:12.566207 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.566266 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:12.566290 52398 natgateway.go:117] DescribeNatGateways: { } I0909 13:49:12.566331 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0909 13:49:12.566363 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0909 13:49:12.566405 52398 api.go:53] PageSize not implemented I0909 13:49:12.566420 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:12.566436 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:12.566447 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0909 13:49:12.566480 52398 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0909 13:49:12.566495 52398 targetgroups.go:35] PageSize not implemented I0909 13:49:12.566506 52398 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I0909 13:49:12.566559 52398 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0909 13:49:12.566589 52398 iamrole.go:89] ListRoles: { } I0909 13:49:12.566630 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.566653 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:12.566686 52398 internetgateways.go:101] DescribeInternetGateways: { } I0909 13:49:12.566727 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:12.566762 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3","rtb-2","rtb-1"] } I0909 13:49:12.566796 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } I0909 13:49:12.566820 52398 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0909 13:49:12.566841 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0909 13:49:12.566864 52398 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-2"] } I0909 13:49:12.567337 52398 egressonlyinternetgateways.go:171] DeleteEgressOnlyInternetGateway: { EgressOnlyInternetGatewayId: "eigw-2" } autoscaling-group:nodes.minimal-ipv6.example.com ok I0909 13:49:12.567395 52398 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0909 13:49:12.567406 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } egress-only-internet-gateway:eigw-2 ok I0909 13:49:12.567439 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0909 13:49:12.567469 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0909 13:49:12.567380 52398 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } iam-instance-profile:masters.minimal-ipv6.example.com ok target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0909 13:49:12.567512 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0909 13:49:12.567531 52398 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I0909 13:49:12.567443 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:12.567546 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } route53-record:Z1AFAKE1ZON3YO/AAAA/api.internal.minimal-ipv6.example.com. ok I0909 13:49:12.567562 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:12.567611 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } iam-instance-profile:nodes.minimal-ipv6.example.com ok I0909 13:49:12.567635 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0909 13:49:12.567987 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:12.568009 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0909 13:49:12.568044 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:12.568098 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0909 13:49:12.568243 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:12.568262 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0909 13:49:12.568296 52398 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0909 13:49:12.568354 52398 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0909 13:49:12.568416 52398 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0909 13:49:12.568451 52398 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-2" } nat-gateway:nat-2 ok I0909 13:49:12.568506 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-6" } subnet:subnet-6 ok autoscaling-config:lt-1 ok I0909 13:49:12.568567 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0909 13:49:12.568617 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:12.568668 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0909 13:49:12.568699 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0909 13:49:12.568820 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-5" } subnet:subnet-5 ok I0909 13:49:12.568835 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0909 13:49:12.568889 52398 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0909 13:49:12.568878 52398 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-2" } elastic-ip:eipalloc-2 ok I0909 13:49:12.568905 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0909 13:49:12.568936 52398 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } I0909 13:49:12.568929 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } iam-role:nodes.minimal-ipv6.example.com ok subnet:subnet-1 ok I0909 13:49:12.568966 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0909 13:49:12.569007 52398 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0909 13:49:12.569020 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0909 13:49:12.569046 52398 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0909 13:49:12.568972 52398 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0909 13:49:12.569106 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0909 13:49:12.569163 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0909 13:49:12.569216 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0909 13:49:12.569248 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0909 13:49:12.569289 52398 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0909 13:49:12.569330 52398 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0909 13:49:12.569512 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.51s) === RUN TestLifecycleSharedVPC I0909 13:49:12.569643 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:12.569731 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:12.569754 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:12.569777 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:12.569800 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:12.569833 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:12.569857 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:12.569875 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:12.569897 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:12.569913 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:12.569942 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0909 13:49:12.586898 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:12.589731 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.589823 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.661179 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.739706 52398 iamrole.go:89] ListRoles: { } I0909 13:49:12.741886 52398 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0909 13:49:12.742170 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:12.742918 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:12.744812 52398 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0909 13:49:12.745194 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:49:12.745227 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0909 13:49:12.745275 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0909 13:49:12.745308 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0909 13:49:12.745305 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0909 13:49:12.745847 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.745930 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0909 13:49:12.746022 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedvpc.example.com" }] } I0909 13:49:12.746057 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:12.746210 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:12.746294 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.746467 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:12.746614 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:12.748241 52398 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0909 13:49:12.748357 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0909 13:49:12.748732 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0909 13:49:12.748969 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" }] }], VpcId: "vpc-12345678" } I0909 13:49:12.749071 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:12.749119 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:12.749191 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:49:12.749417 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0909 13:49:12.749704 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0909 13:49:12.750974 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] }] } I0909 13:49:12.751182 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], VpcId: "vpc-12345678" } I0909 13:49:12.751340 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.751389 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.751468 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0909 13:49:12.751676 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0909 13:49:12.751780 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:12.751832 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:12.751664 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0909 13:49:12.751940 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.752030 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0909 13:49:12.752082 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0909 13:49:12.752167 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] }], VpcId: "vpc-12345678" } I0909 13:49:12.752241 52398 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0909 13:49:12.752258 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:12.752302 52398 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] } I0909 13:49:12.754261 52398 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0909 13:49:12.754497 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" }] }] } I0909 13:49:12.754682 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0909 13:49:12.754816 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.754946 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0909 13:49:12.754983 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0909 13:49:12.755020 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0909 13:49:12.755151 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.755272 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0909 13:49:12.755319 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0909 13:49:12.755523 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0909 13:49:12.755716 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0909 13:49:12.755913 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0909 13:49:12.756043 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0909 13:49:12.756248 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0909 13:49:12.756447 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" }] }] } I0909 13:49:12.756582 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.756698 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0909 13:49:12.756832 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.756927 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.759716 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.760061 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0909 13:49:12.760264 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.761660 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0909 13:49:12.761828 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0909 13:49:12.761979 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0909 13:49:12.762099 52398 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0909 13:49:12.762314 52398 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0909 13:49:12.762405 52398 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0909 13:49:12.762442 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.762474 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:12.762529 52398 dns.go:232] Pre-creating DNS records I0909 13:49:12.762556 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:12.765923 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:12.768187 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.768314 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.839637 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.919106 52398 iamrole.go:89] ListRoles: { } I0909 13:49:12.921024 52398 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0909 13:49:12.921363 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:49:12.921453 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0909 13:49:12.921481 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0909 13:49:12.921331 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:12.921516 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.921681 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:12.921802 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.923321 52398 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0909 13:49:12.923446 52398 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0909 13:49:12.923535 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0909 13:49:12.923502 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.923645 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.923686 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:49:12.923825 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0909 13:49:12.924089 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:12.924169 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.924308 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0909 13:49:12.924495 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:12.927782 52398 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0909 13:49:12.927974 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.928110 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.928268 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.928357 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.928466 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.928599 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:12.928736 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.930402 52398 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0909 13:49:12.930667 52398 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0909 13:49:12.930791 52398 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0909 13:49:12.932317 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0909 13:49:12.932357 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0909 13:49:12.932382 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:12.932404 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0909 13:49:12.932423 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0909 13:49:12.932442 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0909 13:49:12.932464 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0909 13:49:12.932489 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0909 13:49:12.932513 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:12.932531 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:12.932550 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:12.932570 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0909 13:49:12.932589 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:12.932609 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:12.932629 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.932650 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:12.932668 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:12.932687 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0909 13:49:12.932705 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:12.932727 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:12.932748 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0909 13:49:12.933520 52398 delete_cluster.go:128] Looking for cloud resources to delete W0909 13:49:12.933558 52398 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0909 13:49:12.933570 52398 keypairs.go:95] DescribeKeyPairs: { } I0909 13:49:12.933598 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.933628 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0909 13:49:12.933661 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.933701 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.933726 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.933752 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.933779 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.933803 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0909 13:49:12.933848 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.933891 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0909 13:49:12.933913 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0909 13:49:12.933942 52398 api.go:53] PageSize not implemented I0909 13:49:12.933955 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:12.933970 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:12.933982 52398 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0909 13:49:12.933996 52398 targetgroups.go:35] PageSize not implemented I0909 13:49:12.934009 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0909 13:49:12.934042 52398 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0909 13:49:12.934069 52398 iamrole.go:89] ListRoles: { } I0909 13:49:12.934103 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.934121 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:12.934146 52398 internetgateways.go:101] DescribeInternetGateways: { } I0909 13:49:12.934167 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:12.934184 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.sharedvpc.example.com ok I0909 13:49:12.934408 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0909 13:49:12.934454 52398 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0909 13:49:12.934486 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-config:lt-1 ok I0909 13:49:12.934461 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0909 13:49:12.934531 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0909 13:49:12.934520 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I0909 13:49:12.934584 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } W0909 13:49:12.934595 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions iam-instance-profile:masters.sharedvpc.example.com ok I0909 13:49:12.934606 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0909 13:49:12.934606 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I0909 13:49:12.934445 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:12.934632 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedvpc.example.com. ok volume:vol-2 ok I0909 13:49:12.934655 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:12.934680 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0909 13:49:12.934790 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:12.934797 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0909 13:49:12.934830 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0909 13:49:12.934846 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0909 13:49:12.934922 52398 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0909 13:49:12.934943 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0909 13:49:12.934987 52398 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0909 13:49:12.935038 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0909 13:49:12.935097 52398 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0909 13:49:12.935116 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0909 13:49:12.935151 52398 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0909 13:49:12.935212 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0909 13:49:12.935556 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.37s) === RUN TestLifecycleComplex I0909 13:49:12.935786 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:12.935945 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:12.935994 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:12.936052 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:12.936101 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:12.936148 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:12.936189 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:12.936217 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:12.936253 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:12.936281 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:12.936329 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0909 13:49:12.973444 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:49:12.976918 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:12.977015 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.050686 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:13.171361 52398 iamrole.go:89] ListRoles: { } I0909 13:49:13.172602 52398 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0909 13:49:13.172800 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0909 13:49:13.172821 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:13.172804 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.172875 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } W0909 13:49:13.172917 52398 api.go:53] PageSize not implemented I0909 13:49:13.172943 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0909 13:49:13.172970 52398 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" } ] }] } I0909 13:49:13.173039 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0909 13:49:13.173051 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0909 13:49:13.173149 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" } ] } I0909 13:49:13.173108 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0909 13:49:13.173278 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0909 13:49:13.173355 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:13.173398 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0909 13:49:13.173495 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.173614 52398 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0909 13:49:13.173710 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:13.173746 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0909 13:49:13.173807 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.173938 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:13.174032 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:13.174061 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.174152 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/control-plane", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:13.174240 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:13.174269 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.174795 52398 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0909 13:49:13.174885 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.174962 52398 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0909 13:49:13.174996 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0909 13:49:13.174996 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0909 13:49:13.175089 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0909 13:49:13.175072 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I0909 13:49:13.175144 52398 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0909 13:49:13.175185 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.175239 52398 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.timeout_seconds", Value: "30" },{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0909 13:49:13.175299 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0909 13:49:13.175294 52398 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0909 13:49:13.175299 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0909 13:49:13.175341 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.175387 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0909 13:49:13.175442 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0909 13:49:13.175423 52398 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0909 13:49:13.175507 52398 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0909 13:49:13.175500 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.175625 52398 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "complex.example.com" } ] }] } I0909 13:49:13.175690 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.175817 52398 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0909 13:49:13.175853 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.175920 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0909 13:49:13.175999 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0909 13:49:13.175996 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.176079 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0909 13:49:13.176150 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0909 13:49:13.176168 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0909 13:49:13.176236 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0909 13:49:13.176265 52398 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0909 13:49:13.176324 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0909 13:49:13.176194 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0909 13:49:13.176388 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:13.176437 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.176499 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.176543 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0909 13:49:13.176572 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0909 13:49:13.176619 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0909 13:49:13.176690 52398 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0909 13:49:13.176720 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:13.176749 52398 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] } I0909 13:49:13.176801 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:13.176830 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:13.176859 52398 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0909 13:49:13.176890 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:13.176922 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:13.176958 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:13.176985 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:13.177007 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0909 13:49:13.177065 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:13.177095 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:13.177117 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:13.177152 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:13.177194 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.177319 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0909 13:49:13.177394 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0909 13:49:13.177424 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:13.177472 52398 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0909 13:49:13.177537 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.177599 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.177604 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.177715 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.177755 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0909 13:49:13.177802 52398 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-1" }], Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ], Type: "network" } I0909 13:49:13.177892 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:13.177935 52398 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0909 13:49:13.177996 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0909 13:49:13.178033 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0909 13:49:13.178020 52398 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0909 13:49:13.178071 52398 loadbalancers.go:170] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:13.178062 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } W0909 13:49:13.178118 52398 api.go:53] PageSize not implemented I0909 13:49:13.178167 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0909 13:49:13.178306 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0909 13:49:13.178443 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0909 13:49:13.178576 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0909 13:49:13.178705 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0909 13:49:13.178773 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.178822 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0909 13:49:13.178853 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.178890 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.178925 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.178967 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.179025 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" } ] }] } I0909 13:49:13.179112 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0909 13:49:13.179185 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0909 13:49:13.179317 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-1", TransitGatewayId: "tgw-0123456" } I0909 13:49:13.179359 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0909 13:49:13.179459 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0909 13:49:13.179550 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0909 13:49:13.179582 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" } ] }] } I0909 13:49:13.179679 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0909 13:49:13.179709 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0909 13:49:13.179765 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0909 13:49:13.179848 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0909 13:49:13.179900 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" } ] }] } I0909 13:49:13.179981 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0909 13:49:13.180013 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0909 13:49:13.180046 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0909 13:49:13.180067 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0909 13:49:13.180151 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-1", TransitGatewayId: "tgw-123456" } I0909 13:49:13.180180 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0909 13:49:13.180238 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0909 13:49:13.180320 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0909 13:49:13.180404 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0909 13:49:13.180490 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0909 13:49:13.180571 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0909 13:49:13.180655 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0909 13:49:13.180713 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0909 13:49:13.180799 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0909 13:49:13.180906 52398 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0909 13:49:13.180935 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.180970 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.180987 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.180983 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:13.181066 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.181114 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.181092 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:13.181126 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.183939 52398 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0909 13:49:13.184121 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.184198 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.184304 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.184398 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.186109 52398 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0909 13:49:13.186309 52398 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0909 13:49:13.186399 52398 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0909 13:49:13.186435 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:13.186463 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:13.186509 52398 dns.go:232] Pre-creating DNS records I0909 13:49:13.186526 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:13.193256 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:49:13.197094 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.197195 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.268527 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:13.416037 52398 iamrole.go:89] ListRoles: { } I0909 13:49:13.417420 52398 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0909 13:49:13.417636 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0909 13:49:13.417787 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.417839 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0909 13:49:13.417918 52398 api.go:53] PageSize not implemented I0909 13:49:13.417903 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.417966 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:13.417986 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:13.418019 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.418058 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.418225 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.418373 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0909 13:49:13.418483 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0909 13:49:13.418579 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0909 13:49:13.419489 52398 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0909 13:49:13.419596 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0909 13:49:13.419601 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.419670 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0909 13:49:13.419672 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:13.419702 52398 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0909 13:49:13.419706 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0909 13:49:13.419699 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.419800 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:13.419770 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0909 13:49:13.419886 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:13.419920 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0909 13:49:13.419832 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.419976 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.420109 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0909 13:49:13.420259 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:13.420308 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.420438 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:13.420484 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.420607 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:13.420653 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.420738 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0909 13:49:13.420890 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:13.420936 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.421012 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.421085 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0909 13:49:13.421177 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.421271 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:13.421359 52398 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0909 13:49:13.421486 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.421553 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.421537 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.421570 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.421613 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.421604 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.421638 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:13.421661 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:13.421681 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:13.421699 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0909 13:49:13.421755 52398 api.go:53] PageSize not implemented I0909 13:49:13.421789 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.421856 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.421954 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.422026 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.422095 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.422241 52398 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0909 13:49:13.422266 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.422300 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.422316 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.422358 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.422395 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.422406 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.422310 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:13.422477 52398 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0909 13:49:13.422500 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0909 13:49:13.422524 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.426103 52398 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0909 13:49:13.426227 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.426384 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.427899 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.428053 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.429047 52398 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0909 13:49:13.429212 52398 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0909 13:49:13.429307 52398 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0909 13:49:13.433184 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:13.435116 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.441846 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:49:13.444037 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.444092 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.514718 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:13.634717 52398 iamrole.go:89] ListRoles: { } I0909 13:49:13.635922 52398 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0909 13:49:13.636122 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0909 13:49:13.636220 52398 api.go:53] PageSize not implemented I0909 13:49:13.636297 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0909 13:49:13.636445 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.636543 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:13.636555 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0909 13:49:13.636688 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.636848 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:13.636884 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:13.636939 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.636976 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0909 13:49:13.637064 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0909 13:49:13.637151 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.638622 52398 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0909 13:49:13.638742 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.638892 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:13.638997 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.639106 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.639243 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.639361 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:13.639689 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.639828 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0909 13:49:13.639888 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.640022 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.640181 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0909 13:49:13.640319 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:13.640400 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.640542 52398 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0909 13:49:13.641974 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0909 13:49:13.642122 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0909 13:49:13.642166 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.642231 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:13.642270 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:13.642272 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0909 13:49:13.642346 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0909 13:49:13.642360 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0909 13:49:13.642415 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:13.642441 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0909 13:49:13.642457 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:13.642484 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.642567 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:13.642619 52398 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0909 13:49:13.642692 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.642822 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.642885 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.642953 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.642972 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.642964 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.643013 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.643046 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:13.643082 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:13.643120 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:13.643120 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.643150 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:13.643277 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.643360 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.643409 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0909 13:49:13.643441 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:13.643464 52398 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0909 13:49:13.643484 52398 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0909 13:49:13.643502 52398 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0909 13:49:13.643566 52398 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0909 13:49:13.643607 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.643592 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.643630 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } W0909 13:49:13.643656 52398 api.go:53] PageSize not implemented I0909 13:49:13.643951 52398 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0909 13:49:13.643986 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.644033 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.644021 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:13.644074 52398 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0909 13:49:13.644053 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.644157 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.644209 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.644223 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.644281 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0909 13:49:13.644299 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.647065 52398 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0909 13:49:13.647184 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.647342 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.647468 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.647572 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.652406 52398 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0909 13:49:13.653111 52398 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0909 13:49:13.653228 52398 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0909 13:49:13.653268 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:13.653299 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:13.653351 52398 dns.go:232] Pre-creating DNS records I0909 13:49:13.658124 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:49:13.660419 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.660484 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.731847 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:13.881173 52398 iamrole.go:89] ListRoles: { } I0909 13:49:13.882419 52398 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0909 13:49:13.882544 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0909 13:49:13.882616 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:13.882645 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.882781 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } W0909 13:49:13.882814 52398 api.go:53] PageSize not implemented I0909 13:49:13.882820 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:13.882875 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.883033 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0909 13:49:13.883145 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.883188 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0909 13:49:13.883283 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.883408 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0909 13:49:13.883503 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.884459 52398 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0909 13:49:13.884566 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.884593 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0909 13:49:13.884669 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0909 13:49:13.884668 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:13.884692 52398 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0909 13:49:13.884711 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0909 13:49:13.884668 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.884806 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:13.884779 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0909 13:49:13.884854 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:13.884845 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.884890 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0909 13:49:13.884978 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:13.885016 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.885094 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0909 13:49:13.885237 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:13.885280 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.885396 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0909 13:49:13.885542 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:13.885595 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.885716 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.885830 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0909 13:49:13.885963 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:13.886008 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.886085 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:13.886134 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0909 13:49:13.886216 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:13.886273 52398 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0909 13:49:13.886335 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.886414 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.886409 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.886510 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.886503 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.886528 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.886597 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.886599 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.886636 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:13.886673 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:13.886705 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:13.886734 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0909 13:49:13.886820 52398 api.go:53] PageSize not implemented I0909 13:49:13.886820 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.886899 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:13.887026 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:13.887153 52398 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0909 13:49:13.887193 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.887228 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.887238 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.887234 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:13.887307 52398 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0909 13:49:13.887316 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:13.887362 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:13.887378 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.887455 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0909 13:49:13.887481 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:13.891303 52398 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0909 13:49:13.891404 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.891586 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.891748 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.891903 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.894356 52398 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0909 13:49:13.894564 52398 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0909 13:49:13.894692 52398 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0909 13:49:13.898007 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:13.901370 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:13.912881 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:49:13.916501 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.916590 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:13.987708 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:14.126513 52398 iamrole.go:89] ListRoles: { } I0909 13:49:14.127929 52398 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0909 13:49:14.128116 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } W0909 13:49:14.128306 52398 api.go:53] PageSize not implemented I0909 13:49:14.128243 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0909 13:49:14.128377 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:14.128410 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:14.128448 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:14.128512 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.128554 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0909 13:49:14.128638 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0909 13:49:14.128747 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0909 13:49:14.128824 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0909 13:49:14.128935 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0909 13:49:14.129000 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.129106 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.131364 52398 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0909 13:49:14.131497 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.131663 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0909 13:49:14.131625 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.131738 52398 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0909 13:49:14.131683 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0909 13:49:14.131843 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:14.131716 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.131946 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0909 13:49:14.132040 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:14.132091 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0909 13:49:14.132191 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0909 13:49:14.132153 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.132331 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0909 13:49:14.132512 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:14.132573 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:14.132628 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.132710 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.132848 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.132988 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.133133 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:14.133181 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.133278 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.133399 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0909 13:49:14.133517 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0909 13:49:14.133967 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:14.134335 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:14.134463 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:14.134533 52398 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0909 13:49:14.134713 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.134833 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:14.135514 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.135574 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.135588 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.135621 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.135537 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.135643 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:14.135666 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:14.135689 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:14.135709 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0909 13:49:14.135767 52398 api.go:53] PageSize not implemented I0909 13:49:14.135824 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:14.135940 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.136236 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.136423 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:14.136537 52398 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0909 13:49:14.136568 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.136614 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.136643 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.136654 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:14.136768 52398 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0909 13:49:14.136732 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.136796 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.136815 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.136889 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0909 13:49:14.136920 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.139736 52398 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0909 13:49:14.139846 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.139944 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.140033 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.140139 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.142083 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.142158 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.142334 52398 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0909 13:49:14.142498 52398 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0909 13:49:14.142566 52398 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0909 13:49:14.142617 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:14.142663 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:14.142703 52398 dns.go:232] Pre-creating DNS records I0909 13:49:14.148163 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:49:14.153004 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.153098 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.223011 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:14.383108 52398 iamrole.go:89] ListRoles: { } I0909 13:49:14.384973 52398 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0909 13:49:14.385222 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0909 13:49:14.385397 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:14.385447 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0909 13:49:14.385563 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.385691 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0909 13:49:14.385718 52398 api.go:53] PageSize not implemented I0909 13:49:14.385764 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0909 13:49:14.385820 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0909 13:49:14.385875 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.385943 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:14.385964 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:14.385999 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.386022 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.386098 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0909 13:49:14.387380 52398 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0909 13:49:14.387536 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.387657 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0909 13:49:14.387790 52398 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0909 13:49:14.387733 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0909 13:49:14.387892 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:14.387908 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:14.387926 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0909 13:49:14.388004 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0909 13:49:14.388045 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0909 13:49:14.388108 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:14.388133 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0909 13:49:14.388020 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0909 13:49:14.388269 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:14.388331 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.388443 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.388529 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0909 13:49:14.388642 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:14.388732 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.388833 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:14.388859 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.388936 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.388986 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.389063 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:14.389086 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.389136 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.389211 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:14.389233 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.389502 52398 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0909 13:49:14.389775 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:14.389843 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.389941 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.389963 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.390010 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.390050 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:14.390089 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:14.390126 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:14.390088 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.390160 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:14.390267 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0909 13:49:14.390363 52398 api.go:53] PageSize not implemented I0909 13:49:14.390458 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.390707 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.390902 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.391185 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:14.391329 52398 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0909 13:49:14.391383 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.391444 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.391427 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:14.391467 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.391540 52398 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0909 13:49:14.391580 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.391634 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.391654 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.391758 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0909 13:49:14.391807 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.396412 52398 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0909 13:49:14.396547 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.396641 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.396824 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.396962 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.400045 52398 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0909 13:49:14.400269 52398 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0909 13:49:14.400398 52398 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0909 13:49:14.405137 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:14.408457 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.408543 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.421346 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:49:14.423721 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.423791 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.423825 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.495551 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:14.660593 52398 iamrole.go:89] ListRoles: { } I0909 13:49:14.662185 52398 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0909 13:49:14.662314 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0909 13:49:14.662505 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } W0909 13:49:14.662614 52398 api.go:53] PageSize not implemented I0909 13:49:14.662616 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:14.662621 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.662727 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:14.662762 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:14.662806 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0909 13:49:14.662906 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.663016 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0909 13:49:14.663117 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.663217 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.663301 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0909 13:49:14.663357 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.664425 52398 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0909 13:49:14.664544 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0909 13:49:14.664623 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0909 13:49:14.664733 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:14.664762 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0909 13:49:14.664710 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0909 13:49:14.664778 52398 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0909 13:49:14.664807 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0909 13:49:14.664844 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:14.664871 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0909 13:49:14.664867 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:14.664941 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0909 13:49:14.664901 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.665119 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0909 13:49:14.665330 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:14.665512 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:14.665632 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.665767 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.665896 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.666124 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:14.666190 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.666297 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.666442 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.666573 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.666690 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.666820 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.666915 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:14.666974 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:14.667072 52398 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0909 13:49:14.667179 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.667288 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.667333 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.667344 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.667279 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.667392 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.667485 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:14.667524 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:14.667550 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:14.667570 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:14.667523 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0909 13:49:14.667637 52398 api.go:53] PageSize not implemented I0909 13:49:14.667757 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.668019 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:14.668202 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:14.668508 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.668736 52398 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0909 13:49:14.668778 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.668836 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.668825 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:14.668878 52398 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0909 13:49:14.668858 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.668951 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.669055 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.669093 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.669299 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0909 13:49:14.669340 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.677363 52398 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0909 13:49:14.677534 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.677781 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.677984 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.678208 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.680640 52398 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0909 13:49:14.681090 52398 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0909 13:49:14.681236 52398 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0909 13:49:14.681273 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:14.681297 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:14.681339 52398 dns.go:232] Pre-creating DNS records I0909 13:49:14.685969 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:49:14.688367 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.688433 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.688471 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.759323 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:14.877622 52398 iamrole.go:89] ListRoles: { } I0909 13:49:14.878832 52398 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0909 13:49:14.878976 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.879021 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:14.879105 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.879259 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0909 13:49:14.879353 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:14.879376 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:14.879411 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.879439 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0909 13:49:14.879487 52398 api.go:53] PageSize not implemented I0909 13:49:14.879504 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0909 13:49:14.880770 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0909 13:49:14.880870 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0909 13:49:14.881075 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.881190 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0909 13:49:14.882591 52398 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0909 13:49:14.882800 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0909 13:49:14.882837 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0909 13:49:14.882782 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.882959 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:14.882994 52398 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0909 13:49:14.883022 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0909 13:49:14.883269 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0909 13:49:14.883055 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:14.883319 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:14.883340 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0909 13:49:14.883351 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.883436 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0909 13:49:14.883854 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:14.883903 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.883991 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.884078 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:14.884102 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.884156 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0909 13:49:14.884652 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:14.884699 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.884789 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:14.884814 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.884862 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.884935 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:14.884982 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0909 13:49:14.885071 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:14.885099 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:14.885197 52398 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0909 13:49:14.885268 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:14.885396 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.885455 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.885506 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.885520 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.885545 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.885567 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:14.885591 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:14.885572 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:14.885613 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:14.885633 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0909 13:49:14.885691 52398 api.go:53] PageSize not implemented I0909 13:49:14.885732 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.885887 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:14.885971 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.886050 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:14.886127 52398 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0909 13:49:14.886157 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.886192 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.886207 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.886210 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:14.886280 52398 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0909 13:49:14.886289 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:14.886343 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:14.886360 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.886469 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0909 13:49:14.886498 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:14.889263 52398 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0909 13:49:14.889403 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.889517 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.889599 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.889707 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.891191 52398 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0909 13:49:14.891339 52398 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0909 13:49:14.891425 52398 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0909 13:49:14.893096 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:14.895075 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.895128 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.895206 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.901101 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:49:14.903237 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.903291 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.903324 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.903353 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:14.972801 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:15.093602 52398 iamrole.go:89] ListRoles: { } I0909 13:49:15.094872 52398 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0909 13:49:15.095036 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } W0909 13:49:15.095217 52398 api.go:53] PageSize not implemented I0909 13:49:15.096017 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.096437 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.096525 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:15.096569 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:15.096580 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:15.096617 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0909 13:49:15.096777 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0909 13:49:15.096892 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0909 13:49:15.097006 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.097181 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.097235 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0909 13:49:15.097357 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.100512 52398 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0909 13:49:15.100608 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0909 13:49:15.100713 52398 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0909 13:49:15.100719 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:15.100762 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.100815 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0909 13:49:15.100821 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.100791 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0909 13:49:15.101476 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:15.101360 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.101525 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0909 13:49:15.101592 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:15.101594 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0909 13:49:15.101623 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.101650 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:15.101675 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0909 13:49:15.101677 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0909 13:49:15.101783 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:15.101837 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.101992 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:15.102038 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.102133 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0909 13:49:15.102300 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:15.102351 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.102488 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.102622 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.102711 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.102847 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:15.102938 52398 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0909 13:49:15.103094 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.103169 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:15.103215 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:15.103259 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:15.103271 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:15.103237 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:15.103298 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:15.103320 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:15.103345 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:15.103366 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:15.103384 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0909 13:49:15.103438 52398 api.go:53] PageSize not implemented I0909 13:49:15.103551 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.103804 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:15.104178 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:15.104377 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.104503 52398 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0909 13:49:15.104541 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:15.104574 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:15.104584 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:15.104571 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:15.104649 52398 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0909 13:49:15.104665 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:15.104711 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:15.104726 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:15.104815 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0909 13:49:15.104848 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:15.107408 52398 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0909 13:49:15.107472 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.107553 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.107673 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.107705 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.109621 52398 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0909 13:49:15.110076 52398 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0909 13:49:15.110243 52398 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0909 13:49:15.115789 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:15.115854 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:15.115929 52398 dns.go:232] Pre-creating DNS records I0909 13:49:15.120911 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0909 13:49:15.123257 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.123319 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.123353 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.123383 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.194498 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:15.327475 52398 iamrole.go:89] ListRoles: { } I0909 13:49:15.328740 52398 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0909 13:49:15.329676 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0909 13:49:15.329829 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0909 13:49:15.329893 52398 api.go:53] PageSize not implemented I0909 13:49:15.329925 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:15.329947 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:15.330008 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.330099 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:15.330116 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0909 13:49:15.330286 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0909 13:49:15.330651 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.330887 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.331288 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0909 13:49:15.331463 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0909 13:49:15.331582 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0909 13:49:15.334478 52398 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0909 13:49:15.334559 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0909 13:49:15.334624 52398 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0909 13:49:15.334594 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.334693 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0909 13:49:15.334760 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:15.334724 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.334794 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0909 13:49:15.334842 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0909 13:49:15.334922 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:15.334957 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0909 13:49:15.334844 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0909 13:49:15.335082 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:15.335157 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.335246 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.335337 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.335388 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0909 13:49:15.335499 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:15.335581 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.335696 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.335753 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.335803 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0909 13:49:15.335896 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:15.335924 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.336004 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0909 13:49:15.336028 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:15.336051 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0909 13:49:15.336122 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:15.336169 52398 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0909 13:49:15.336269 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:15.336305 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:15.336352 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:15.336363 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:15.336353 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:15.336386 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:15.336408 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0909 13:49:15.336404 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.336431 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0909 13:49:15.336492 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0909 13:49:15.336524 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0909 13:49:15.336584 52398 api.go:53] PageSize not implemented I0909 13:49:15.336585 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.336680 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:15.336777 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.336889 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:15.337013 52398 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0909 13:49:15.337039 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:15.337069 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:15.337081 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:15.337083 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:15.337138 52398 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0909 13:49:15.337150 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:15.337195 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:15.337210 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:15.337281 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0909 13:49:15.337315 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:15.340104 52398 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0909 13:49:15.340163 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.340259 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.340327 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.340445 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.342177 52398 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0909 13:49:15.342307 52398 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0909 13:49:15.342389 52398 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0909 13:49:15.342490 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0909 13:49:15.342522 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:15.342546 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0909 13:49:15.342565 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:15.342584 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0909 13:49:15.342603 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0909 13:49:15.342624 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:15.342647 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0909 13:49:15.342669 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0909 13:49:15.342693 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:15.342714 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:15.342734 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:15.342755 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0909 13:49:15.342774 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0909 13:49:15.342795 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:15.342815 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:15.342836 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:15.342857 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:15.342880 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:15.342899 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:15.342920 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0909 13:49:15.342941 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:15.342959 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0909 13:49:15.342979 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:15.343000 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:15.343021 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:15.343048 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0909 13:49:15.344539 52398 delete_cluster.go:128] Looking for cloud resources to delete W0909 13:49:15.344588 52398 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0909 13:49:15.344608 52398 keypairs.go:95] DescribeKeyPairs: { } I0909 13:49:15.344633 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.344682 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0909 13:49:15.344744 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.344801 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.344866 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.344921 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.344964 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.345010 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0909 13:49:15.345048 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.345097 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0909 13:49:15.345124 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0909 13:49:15.345159 52398 api.go:53] PageSize not implemented I0909 13:49:15.345172 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:15.345187 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:15.345198 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0909 13:49:15.345223 52398 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0909 13:49:15.345237 52398 targetgroups.go:35] PageSize not implemented I0909 13:49:15.345248 52398 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I0909 13:49:15.345299 52398 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0909 13:49:15.345324 52398 iamrole.go:89] ListRoles: { } I0909 13:49:15.345355 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:15.345374 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:15.345403 52398 internetgateways.go:101] DescribeInternetGateways: { } I0909 13:49:15.345435 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:15.345462 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0909 13:49:15.345782 52398 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5 ok autoscaling-group:nodes.complex.example.com ok autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0909 13:49:15.345819 52398 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } autoscaling-config:lt-1 ok I0909 13:49:15.345891 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0909 13:49:15.345845 52398 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I0909 13:49:15.345909 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0909 13:49:15.345936 52398 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0909 13:49:15.345957 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0909 13:49:15.345965 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0909 13:49:15.345956 52398 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0909 13:49:15.345879 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:15.345980 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4 ok route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.complex.example.com. ok I0909 13:49:15.346007 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok I0909 13:49:15.345998 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:15.346034 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 } ] } W0909 13:49:15.346185 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:15.346200 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0909 13:49:15.346232 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:15.346265 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0909 13:49:15.346472 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:15.346485 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0909 13:49:15.346506 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0909 13:49:15.346543 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:15.346575 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0909 13:49:15.346595 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0909 13:49:15.346652 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0909 13:49:15.346682 52398 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0909 13:49:15.346658 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0909 13:49:15.346693 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0909 13:49:15.346719 52398 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0909 13:49:15.346720 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0909 13:49:15.346739 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0909 13:49:15.346763 52398 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0909 13:49:15.346771 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0909 13:49:15.346806 52398 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0909 13:49:15.346842 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0909 13:49:15.346893 52398 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0909 13:49:15.346938 52398 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0909 13:49:15.347137 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.41s) === RUN TestLifecycleExternalLB I0909 13:49:15.347314 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:15.347403 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:15.347427 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:15.347453 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:15.347475 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:15.347510 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:15.347537 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:15.347555 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:15.347577 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:15.347594 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:15.347624 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0909 13:49:15.362499 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:15.364226 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.364288 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.435361 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:15.535114 52398 iamrole.go:89] ListRoles: { } I0909 13:49:15.536632 52398 executor.go:111] Tasks: 0 done / 93 total; 49 can run I0909 13:49:15.536871 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.537137 52398 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0909 13:49:15.537290 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0909 13:49:15.537362 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0909 13:49:15.537435 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:15.537515 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.538816 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0909 13:49:15.538905 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0909 13:49:15.539007 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I0909 13:49:15.539150 52398 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0909 13:49:15.539251 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:15.539620 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0909 13:49:15.539739 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0909 13:49:15.539809 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" } I0909 13:49:15.539928 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0909 13:49:15.540029 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0909 13:49:15.540095 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" } W0909 13:49:15.541264 52398 api.go:53] PageSize not implemented W0909 13:49:15.541480 52398 api.go:53] PageSize not implemented I0909 13:49:15.541615 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0909 13:49:15.541692 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" }] } I0909 13:49:15.542750 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0909 13:49:15.542743 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.542878 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0909 13:49:15.542927 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" } W0909 13:49:15.543113 52398 api.go:53] PageSize not implemented I0909 13:49:15.543002 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:15.543006 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:15.543187 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:15.543239 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.543390 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:15.543513 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:15.543555 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:15.543689 52398 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" }] }] } I0909 13:49:15.544602 52398 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0909 13:49:15.544710 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.544818 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0909 13:49:15.544807 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0909 13:49:15.544921 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0909 13:49:15.544867 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0909 13:49:15.544976 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0909 13:49:15.545068 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0909 13:49:15.545006 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0909 13:49:15.545092 52398 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0909 13:49:15.545164 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "masters.externallb.example.com" }] }], VpcId: "vpc-1" } I0909 13:49:15.545262 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:15.545260 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0909 13:49:15.545317 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:15.545347 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.545461 52398 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0909 13:49:15.545493 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.545595 52398 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0909 13:49:15.545646 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:15.545673 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.545752 52398 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0909 13:49:15.545793 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.545952 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:15.545938 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0909 13:49:15.546003 52398 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0909 13:49:15.546061 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:15.546133 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:15.546165 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:15.546200 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:15.546242 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:15.546872 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0909 13:49:15.547673 52398 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0909 13:49:15.547803 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0909 13:49:15.547906 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.548048 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0909 13:49:15.548104 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0909 13:49:15.548359 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.548465 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.548612 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0909 13:49:15.548842 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0909 13:49:15.548977 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" }] }] } I0909 13:49:15.549129 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" }] }] } I0909 13:49:15.549270 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.549393 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.549498 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" }] }] } I0909 13:49:15.549691 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" }] }] } I0909 13:49:15.549894 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0909 13:49:15.550093 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0909 13:49:15.550174 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" }] }] } I0909 13:49:15.550377 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.550561 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.550658 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0909 13:49:15.550845 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0909 13:49:15.551083 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0909 13:49:15.551314 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0909 13:49:15.551542 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0909 13:49:15.551657 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0909 13:49:15.552096 52398 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0909 13:49:15.552314 52398 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0909 13:49:15.552410 52398 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0909 13:49:15.552448 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:15.552479 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:15.552536 52398 dns.go:232] Pre-creating DNS records I0909 13:49:15.552561 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:15.557626 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:15.560436 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.560560 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.632666 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:15.731571 52398 iamrole.go:89] ListRoles: { } I0909 13:49:15.732728 52398 executor.go:111] Tasks: 0 done / 93 total; 49 can run I0909 13:49:15.732871 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0909 13:49:15.732985 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0909 13:49:15.733021 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" } W0909 13:49:15.733059 52398 api.go:53] PageSize not implemented W0909 13:49:15.733130 52398 api.go:53] PageSize not implemented I0909 13:49:15.733086 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:15.733105 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0909 13:49:15.733041 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] }] } I0909 13:49:15.733170 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } W0909 13:49:15.733196 52398 api.go:53] PageSize not implemented I0909 13:49:15.733211 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" } I0909 13:49:15.733214 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I0909 13:49:15.733268 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0909 13:49:15.733344 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:15.733351 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0909 13:49:15.733390 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" } I0909 13:49:15.733378 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:15.733476 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.733565 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.733651 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:15.733685 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.734674 52398 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0909 13:49:15.734814 52398 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0909 13:49:15.734756 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0909 13:49:15.734886 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0909 13:49:15.734887 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.735031 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:15.735077 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:15.735133 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.735234 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.735286 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:15.735331 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.735405 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0909 13:49:15.735493 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:15.737403 52398 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0909 13:49:15.737578 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.737708 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.737810 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.737977 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.738183 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.738307 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.738397 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.739627 52398 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0909 13:49:15.739791 52398 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0909 13:49:15.739916 52398 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0909 13:49:15.742583 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0909 13:49:15.742633 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:15.742667 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0909 13:49:15.742699 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:15.742730 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0909 13:49:15.742761 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0909 13:49:15.742792 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:15.742825 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0909 13:49:15.742860 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0909 13:49:15.742898 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0909 13:49:15.742932 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:15.742962 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:15.742992 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:15.743024 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0909 13:49:15.743053 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:15.743086 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:15.743118 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:15.743153 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:15.743184 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:15.743214 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0909 13:49:15.743244 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:15.743278 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:15.743312 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:15.743345 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0909 13:49:15.744556 52398 delete_cluster.go:128] Looking for cloud resources to delete W0909 13:49:15.744612 52398 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0909 13:49:15.744629 52398 keypairs.go:95] DescribeKeyPairs: { } I0909 13:49:15.744666 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.744713 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0909 13:49:15.744762 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.744815 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.744861 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.744911 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.744950 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.744989 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0909 13:49:15.745034 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.745081 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0909 13:49:15.745118 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0909 13:49:15.745164 52398 api.go:53] PageSize not implemented I0909 13:49:15.745182 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:15.745208 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:15.745223 52398 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0909 13:49:15.745245 52398 targetgroups.go:35] PageSize not implemented I0909 13:49:15.745262 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0909 13:49:15.745311 52398 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0909 13:49:15.745349 52398 iamrole.go:89] ListRoles: { } I0909 13:49:15.745391 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:15.745417 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:15.745453 52398 internetgateways.go:101] DescribeInternetGateways: { } I0909 13:49:15.745490 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:15.745517 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:15.745897 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-config:lt-1 ok I0909 13:49:15.745917 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I0909 13:49:15.745980 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0909 13:49:15.745960 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } volume:vol-1 ok I0909 13:49:15.746025 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0909 13:49:15.746066 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } W0909 13:49:15.746110 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:15.745995 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:15.746124 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.externallb.example.com. ok I0909 13:49:15.746115 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } autoscaling-config:lt-2 ok autoscaling-group:nodes.externallb.example.com ok security-group:sg-2 ok I0909 13:49:15.746154 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } iam-instance-profile:masters.externallb.example.com ok volume:vol-2 ok I0909 13:49:15.746184 52398 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0909 13:49:15.746221 52398 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0909 13:49:15.746248 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:15.746282 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0909 13:49:15.746404 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:15.746418 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0909 13:49:15.746439 52398 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0909 13:49:15.746484 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0909 13:49:15.746514 52398 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0909 13:49:15.746499 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0909 13:49:15.746523 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0909 13:49:15.746562 52398 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0909 13:49:15.746592 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0909 13:49:15.746632 52398 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0909 13:49:15.746644 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0909 13:49:15.746670 52398 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0909 13:49:15.746721 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0909 13:49:15.746789 52398 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0909 13:49:15.746851 52398 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0909 13:49:15.747073 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.40s) === RUN TestLifecycleSharedSubnet I0909 13:49:15.747270 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:15.747379 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:15.747409 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:15.747445 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:15.747482 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:15.747529 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:15.747564 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:15.747592 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:15.747626 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:15.747652 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:15.747697 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0909 13:49:15.769393 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:15.771904 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.771988 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.843630 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:15.904750 52398 iamrole.go:89] ListRoles: { } I0909 13:49:15.906419 52398 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0909 13:49:15.906617 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:15.907359 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:15.907353 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:15.907535 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:15.907578 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0909 13:49:15.907605 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:49:15.907674 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0909 13:49:15.907702 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0909 13:49:15.907665 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0909 13:49:15.907764 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0909 13:49:15.907762 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:15.907821 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0909 13:49:15.907919 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:15.908170 52398 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0909 13:49:15.908425 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:15.908605 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:15.909914 52398 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0909 13:49:15.910000 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0909 13:49:15.910094 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0909 13:49:15.910127 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0909 13:49:15.910251 52398 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0909 13:49:15.910303 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0909 13:49:15.910871 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0909 13:49:15.910250 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:49:15.911025 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:15.911090 52398 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" } ] } I0909 13:49:15.911184 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0909 13:49:15.911308 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0909 13:49:15.911354 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0909 13:49:15.911445 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0909 13:49:15.911516 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:15.911556 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:15.911599 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:15.911662 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:15.912078 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0909 13:49:15.913754 52398 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0909 13:49:15.913918 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.913992 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.914036 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0909 13:49:15.914194 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0909 13:49:15.914280 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0909 13:49:15.914400 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0909 13:49:15.914480 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0909 13:49:15.914600 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0909 13:49:15.914806 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0909 13:49:15.914932 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0909 13:49:15.915106 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0909 13:49:15.915201 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" }] }] } I0909 13:49:15.915303 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0909 13:49:15.915412 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0909 13:49:15.915506 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" }] }] } I0909 13:49:15.915591 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" }] }] } I0909 13:49:15.915667 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.915714 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.917074 52398 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0909 13:49:15.917261 52398 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0909 13:49:15.917338 52398 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0909 13:49:15.917367 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:15.917390 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:15.917430 52398 dns.go:232] Pre-creating DNS records I0909 13:49:15.917448 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:15.921447 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:15.924014 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.924100 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:15.995478 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:16.083266 52398 iamrole.go:89] ListRoles: { } I0909 13:49:16.084568 52398 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0909 13:49:16.084823 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:16.085121 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:49:16.085164 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:16.085293 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0909 13:49:16.086708 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0909 13:49:16.086753 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:16.087083 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:16.089078 52398 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0909 13:49:16.089260 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0909 13:49:16.089424 52398 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0909 13:49:16.089268 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0909 13:49:16.089571 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:49:16.090533 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0909 13:49:16.090672 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0909 13:49:16.091372 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:16.091427 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:16.094689 52398 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0909 13:49:16.095000 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.095185 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.095408 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.095601 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.097370 52398 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0909 13:49:16.097592 52398 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0909 13:49:16.097738 52398 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0909 13:49:16.100238 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0909 13:49:16.100298 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0909 13:49:16.100334 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:16.100367 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0909 13:49:16.100401 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0909 13:49:16.100435 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0909 13:49:16.100472 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0909 13:49:16.100535 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0909 13:49:16.100585 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:16.100619 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:16.100649 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0909 13:49:16.100680 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:16.100715 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:16.100750 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:16.100787 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:16.100822 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0909 13:49:16.100854 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:16.100892 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:16.100928 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0909 13:49:16.102247 52398 delete_cluster.go:128] Looking for cloud resources to delete W0909 13:49:16.102307 52398 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0909 13:49:16.102326 52398 keypairs.go:95] DescribeKeyPairs: { } I0909 13:49:16.102363 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:16.102416 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0909 13:49:16.102471 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:16.102529 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:16.102570 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:16.102611 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:16.102688 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:16.102755 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0909 13:49:16.102824 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:16.102900 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0909 13:49:16.102970 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0909 13:49:16.103058 52398 api.go:53] PageSize not implemented I0909 13:49:16.103100 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:16.103149 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:16.103191 52398 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0909 13:49:16.103237 52398 targetgroups.go:35] PageSize not implemented I0909 13:49:16.103276 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0909 13:49:16.103366 52398 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0909 13:49:16.103434 52398 iamrole.go:89] ListRoles: { } I0909 13:49:16.103510 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:16.103566 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:16.103634 52398 internetgateways.go:101] DescribeInternetGateways: { } I0909 13:49:16.103672 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:16.104113 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok autoscaling-group:nodes.sharedsubnet.example.com ok I0909 13:49:16.104052 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedsubnet.example.com. ok I0909 13:49:16.104253 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0909 13:49:16.104210 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0909 13:49:16.104350 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok W0909 13:49:16.104366 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:16.104406 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I0909 13:49:16.104404 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0909 13:49:16.104453 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:16.104497 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I0909 13:49:16.104513 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0909 13:49:16.104724 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:16.104753 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0909 13:49:16.104786 52398 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0909 13:49:16.104872 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0909 13:49:16.104954 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0909 13:49:16.105037 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0909 13:49:16.105092 52398 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0909 13:49:16.105111 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0909 13:49:16.105146 52398 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0909 13:49:16.105186 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0909 13:49:16.105230 52398 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0909 13:49:16.105244 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0909 13:49:16.105274 52398 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0909 13:49:16.105525 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.36s) === RUN TestLifecyclePrivateSharedSubnet I0909 13:49:16.105706 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:16.105873 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:16.105921 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:16.105987 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:16.106044 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:16.106124 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:16.106188 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:16.106232 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:16.106290 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:16.106336 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:16.106409 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0909 13:49:16.136462 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:16.139616 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.139701 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.139766 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.211301 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:16.305078 52398 iamrole.go:89] ListRoles: { } I0909 13:49:16.306936 52398 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0909 13:49:16.307377 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:49:16.307492 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0909 13:49:16.307454 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:16.307531 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0909 13:49:16.307577 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0909 13:49:16.307603 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0909 13:49:16.307722 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:16.307731 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0909 13:49:16.307880 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0909 13:49:16.307886 52398 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.308003 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0909 13:49:16.308270 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0909 13:49:16.308181 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:16.308403 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0909 13:49:16.308504 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:16.308581 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] }] } I0909 13:49:16.308704 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:16.308794 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:16.309816 52398 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0909 13:49:16.309956 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0909 13:49:16.310152 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0909 13:49:16.310152 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0909 13:49:16.310205 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-su-5ol32q"] } I0909 13:49:16.310320 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0909 13:49:16.310322 52398 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-private-shared-su-5ol32q", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-private-shared-su-5ol32q" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-12345678" } I0909 13:49:16.310406 52398 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0909 13:49:16.310425 52398 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.timeout_seconds", Value: "30" },{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } I0909 13:49:16.310438 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0909 13:49:16.310345 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0909 13:49:16.310494 52398 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0909 13:49:16.310606 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0909 13:49:16.310666 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0909 13:49:16.310650 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0909 13:49:16.310757 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:16.310751 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0909 13:49:16.310820 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0909 13:49:16.310990 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0909 13:49:16.311052 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:16.311084 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:16.311112 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:16.311147 52398 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] } I0909 13:49:16.311202 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0909 13:49:16.311349 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0909 13:49:16.311408 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:16.311437 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:16.311462 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0909 13:49:16.311533 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:16.311559 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:49:16.311625 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:16.311657 52398 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" } ] } I0909 13:49:16.311701 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0909 13:49:16.311750 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0909 13:49:16.311775 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0909 13:49:16.312049 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0909 13:49:16.312523 52398 executor.go:111] Tasks: 67 done / 107 total; 27 can run I0909 13:49:16.312699 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0909 13:49:16.312788 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0909 13:49:16.312801 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:16.312886 52398 loadbalancers.go:35] PageSize not implemented W0909 13:49:16.312910 52398 api.go:53] PageSize not implemented I0909 13:49:16.313003 52398 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-private-shared-su-5ol32q", SubnetMappings: [{ SubnetId: "subnet-abcdef" }], Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }], Type: "network" } I0909 13:49:16.312910 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.313118 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.313245 52398 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0909 13:49:16.313357 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.313207 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0909 13:49:16.313412 52398 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0909 13:49:16.313511 52398 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0909 13:49:16.313584 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.313786 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0909 13:49:16.314055 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0909 13:49:16.314231 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.314422 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0909 13:49:16.314677 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0909 13:49:16.314824 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0909 13:49:16.314876 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0909 13:49:16.314917 52398 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1", Port: 22, Protocol: "TCP" } W0909 13:49:16.315053 52398 api.go:53] PageSize not implemented I0909 13:49:16.315087 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.315020 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0909 13:49:16.315184 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.315372 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0909 13:49:16.315504 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.315674 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.315841 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.316015 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.316190 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.316364 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.316500 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.316573 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.316772 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0909 13:49:16.316881 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0909 13:49:16.317029 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0909 13:49:16.317189 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" }] }] } I0909 13:49:16.317411 52398 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0909 13:49:16.317543 52398 api.go:53] PageSize not implemented I0909 13:49:16.317511 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:16.317602 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.317679 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } W0909 13:49:16.317784 52398 api.go:53] PageSize not implemented I0909 13:49:16.317831 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.322506 52398 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0909 13:49:16.322813 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.322933 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.323033 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.323158 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.325272 52398 executor.go:111] Tasks: 103 done / 107 total; 2 can run I0909 13:49:16.325541 52398 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0909 13:49:16.325705 52398 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0909 13:49:16.325761 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:16.325797 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:16.325862 52398 dns.go:232] Pre-creating DNS records I0909 13:49:16.325887 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:16.331762 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:16.334706 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.334790 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.334851 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.407982 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:16.501657 52398 iamrole.go:89] ListRoles: { } I0909 13:49:16.503748 52398 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0909 13:49:16.504041 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:16.504074 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0909 13:49:16.505199 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:49:16.505264 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0909 13:49:16.505284 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0909 13:49:16.505341 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:16.505919 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] }] } I0909 13:49:16.510462 52398 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0909 13:49:16.510698 52398 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0909 13:49:16.510755 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0909 13:49:16.510810 52398 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0909 13:49:16.510767 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0909 13:49:16.510883 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0909 13:49:16.510898 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-su-5ol32q"] } I0909 13:49:16.511023 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:16.511020 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4"] } I0909 13:49:16.511069 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } I0909 13:49:16.511065 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0909 13:49:16.512151 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0909 13:49:16.512221 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:49:16.512351 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0909 13:49:16.512502 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0909 13:49:16.512673 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0909 13:49:16.512819 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:16.512903 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:16.513022 52398 executor.go:111] Tasks: 67 done / 107 total; 27 can run W0909 13:49:16.513379 52398 api.go:53] PageSize not implemented I0909 13:49:16.513380 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:16.513456 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:16.513414 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.513474 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0909 13:49:16.513507 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0909 13:49:16.513510 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.513531 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } I0909 13:49:16.513554 52398 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0909 13:49:16.513562 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4"] } I0909 13:49:16.513593 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } W0909 13:49:16.513645 52398 api.go:53] PageSize not implemented I0909 13:49:16.513659 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.513547 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.513804 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.514046 52398 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0909 13:49:16.514204 52398 api.go:53] PageSize not implemented I0909 13:49:16.514174 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:16.514307 52398 dnsname.go:118] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0909 13:49:16.514277 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W0909 13:49:16.514475 52398 api.go:53] PageSize not implemented I0909 13:49:16.514517 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.514602 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.517301 52398 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0909 13:49:16.517426 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.517522 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.517744 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.517881 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.520056 52398 executor.go:111] Tasks: 103 done / 107 total; 2 can run W0909 13:49:16.520160 52398 api.go:53] PageSize not implemented I0909 13:49:16.520183 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.520370 52398 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0909 13:49:16.520474 52398 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0909 13:49:16.522429 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0909 13:49:16.522468 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0909 13:49:16.522491 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:16.522510 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0909 13:49:16.522530 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0909 13:49:16.522549 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0909 13:49:16.522569 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0909 13:49:16.522594 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0909 13:49:16.522615 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0909 13:49:16.522641 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:16.522663 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:16.522682 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0909 13:49:16.522702 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:16.522723 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:16.522746 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:16.522766 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0909 13:49:16.522787 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:16.522822 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:16.522851 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0909 13:49:16.522882 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:16.522916 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:16.522951 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0909 13:49:16.524237 52398 delete_cluster.go:128] Looking for cloud resources to delete W0909 13:49:16.524281 52398 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0909 13:49:16.524294 52398 keypairs.go:95] DescribeKeyPairs: { } I0909 13:49:16.524320 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0909 13:49:16.524357 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0909 13:49:16.524400 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0909 13:49:16.524443 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0909 13:49:16.524467 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0909 13:49:16.524500 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0909 13:49:16.524527 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0909 13:49:16.524551 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0909 13:49:16.524574 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0909 13:49:16.524604 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0909 13:49:16.524625 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0909 13:49:16.524659 52398 api.go:53] PageSize not implemented I0909 13:49:16.524670 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0909 13:49:16.524697 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:16.524712 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:16.524723 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0909 13:49:16.524745 52398 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0909 13:49:16.524759 52398 targetgroups.go:35] PageSize not implemented I0909 13:49:16.524770 52398 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" ] } I0909 13:49:16.524811 52398 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0909 13:49:16.524838 52398 iamrole.go:89] ListRoles: { } I0909 13:49:16.524875 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:16.524894 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:16.524920 52398 internetgateways.go:101] DescribeInternetGateways: { } I0909 13:49:16.524949 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:16.525259 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0909 13:49:16.525302 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } I0909 13:49:16.525331 52398 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:bastion.private-shared-subnet.example.com ok I0909 13:49:16.525365 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } keypair:key-1 ok I0909 13:49:16.525335 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0909 13:49:16.525422 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0909 13:49:16.525411 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } volume:vol-2 ok autoscaling-config:lt-3 ok I0909 13:49:16.525439 52398 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok I0909 13:49:16.525462 52398 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4 ok autoscaling-group:nodes.private-shared-subnet.example.com ok I0909 13:49:16.525478 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0909 13:49:16.525591 52398 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1 ok I0909 13:49:16.525622 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0909 13:49:16.525362 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-subnet.example.com. ok I0909 13:49:16.525686 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0909 13:49:16.525523 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 } ] } I0909 13:49:16.525772 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } W0909 13:49:16.525795 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions iam-instance-profile:nodes.private-shared-subnet.example.com ok I0909 13:49:16.525824 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0909 13:49:16.525868 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0909 13:49:16.525995 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:16.526016 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0909 13:49:16.526049 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:16.526105 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0909 13:49:16.526293 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:16.526311 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0909 13:49:16.526404 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0909 13:49:16.526467 52398 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0909 13:49:16.526485 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0909 13:49:16.526464 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:16.526521 52398 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0909 13:49:16.526573 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0909 13:49:16.526547 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } I0909 13:49:16.526663 52398 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com W0909 13:49:16.526666 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:16.526683 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0909 13:49:16.526680 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0909 13:49:16.526715 52398 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0909 13:49:16.526754 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0909 13:49:16.526799 52398 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0909 13:49:16.526813 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0909 13:49:16.526843 52398 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0909 13:49:16.527136 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.42s) === RUN TestLifecyclePrivateSharedIP I0909 13:49:16.527352 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:16.527491 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:16.527545 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:16.527595 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:16.527634 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:16.527682 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:16.527725 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:16.527755 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:16.527793 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:16.527821 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:16.527875 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0909 13:49:16.553318 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:16.555656 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.555727 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.555771 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.629699 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:16.696865 52398 iamrole.go:89] ListRoles: { } I0909 13:49:16.698086 52398 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0909 13:49:16.698425 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.698566 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:16.698559 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0909 13:49:16.698663 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:16.698698 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0909 13:49:16.698832 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:16.698893 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.698942 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0909 13:49:16.699022 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0909 13:49:16.699051 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:16.699148 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:16.699129 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0909 13:49:16.699175 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:16.699185 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0909 13:49:16.699273 52398 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.699379 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:49:16.699425 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0909 13:49:16.699442 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0909 13:49:16.700427 52398 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0909 13:49:16.700657 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0909 13:49:16.700580 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0909 13:49:16.700689 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0909 13:49:16.700765 52398 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0909 13:49:16.700726 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-ip-eepmph"] } I0909 13:49:16.700714 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0909 13:49:16.700868 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0909 13:49:16.700857 52398 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-private-shared-ip-eepmph", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-private-shared-ip-eepmph" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-12345678" } I0909 13:49:16.700933 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0909 13:49:16.700983 52398 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0909 13:49:16.700961 52398 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } I0909 13:49:16.700954 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0909 13:49:16.701036 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:16.701071 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:16.701047 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0909 13:49:16.701103 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0909 13:49:16.701101 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.701168 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0909 13:49:16.701241 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0909 13:49:16.701245 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0909 13:49:16.701400 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0909 13:49:16.701467 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:16.701510 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:16.701544 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0909 13:49:16.701698 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0909 13:49:16.701762 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:16.701791 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:16.701819 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:49:16.701890 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.701947 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0909 13:49:16.702017 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0909 13:49:16.702024 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0909 13:49:16.702099 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0909 13:49:16.702138 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0909 13:49:16.702165 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.702303 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] }] } I0909 13:49:16.702288 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0909 13:49:16.702409 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:16.702439 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:16.702472 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0909 13:49:16.702441 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0909 13:49:16.702503 52398 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] } I0909 13:49:16.702560 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:16.702591 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:16.702613 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:16.702637 52398 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" } ] } I0909 13:49:16.702554 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0909 13:49:16.702956 52398 executor.go:111] Tasks: 69 done / 116 total; 32 can run I0909 13:49:16.703079 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0909 13:49:16.703221 52398 api.go:53] PageSize not implemented I0909 13:49:16.703204 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:16.703255 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:16.703152 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" }] }] } I0909 13:49:16.703302 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.703322 52398 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-private-shared-ip-eepmph", SubnetMappings: [{ SubnetId: "subnet-2" }], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }], Type: "network" } I0909 13:49:16.703354 52398 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-ip.example.com" }] } I0909 13:49:16.703338 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0909 13:49:16.703391 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.703411 52398 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0909 13:49:16.703406 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0909 13:49:16.703443 52398 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0909 13:49:16.703532 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" }] }] } I0909 13:49:16.703617 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.703745 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" }] }] } I0909 13:49:16.703896 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0909 13:49:16.704018 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.704086 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.704152 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:16.704240 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0909 13:49:16.704339 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.private-shared-ip.example.com" }] }] } I0909 13:49:16.704452 52398 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0909 13:49:16.704514 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0909 13:49:16.704549 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.704632 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.704732 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.704855 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.704938 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.705035 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:16.705083 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:16.705123 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.705207 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:16.705229 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0909 13:49:16.705290 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.705295 52398 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1", Port: 22, Protocol: "TCP" } I0909 13:49:16.705370 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } W0909 13:49:16.705395 52398 api.go:53] PageSize not implemented I0909 13:49:16.705413 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.705454 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:16.705483 52398 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0909 13:49:16.705511 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.705590 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.705671 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.705765 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.705884 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.705971 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0909 13:49:16.706028 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.706108 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0909 13:49:16.706137 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0909 13:49:16.706162 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0909 13:49:16.706182 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0909 13:49:16.706205 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.706312 52398 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0909 13:49:16.706359 52398 api.go:53] PageSize not implemented I0909 13:49:16.706357 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:16.706378 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:16.706401 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.706442 52398 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:16.706442 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } W0909 13:49:16.706537 52398 api.go:53] PageSize not implemented I0909 13:49:16.706521 52398 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0909 13:49:16.706583 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0909 13:49:16.706614 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:16.706639 52398 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0909 13:49:16.706568 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.711151 52398 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0909 13:49:16.711217 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:16.711310 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0909 13:49:16.711395 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.711456 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.711539 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.711640 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.714053 52398 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0909 13:49:16.714253 52398 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0909 13:49:16.714354 52398 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0909 13:49:16.714388 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:16.714412 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:16.714452 52398 dns.go:232] Pre-creating DNS records I0909 13:49:16.714468 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:16.718015 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:16.719847 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.719899 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.719936 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.790086 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:16.856039 52398 iamrole.go:89] ListRoles: { } I0909 13:49:16.857620 52398 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0909 13:49:16.857829 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:16.857847 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.857959 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0909 13:49:16.858053 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0909 13:49:16.858092 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0909 13:49:16.858137 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:16.858264 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.859733 52398 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0909 13:49:16.859830 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0909 13:49:16.859885 52398 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0909 13:49:16.859817 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.859910 52398 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0909 13:49:16.859944 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0909 13:49:16.860043 52398 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-ip-eepmph"] } I0909 13:49:16.860103 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:16.860115 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4"] } I0909 13:49:16.860147 52398 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } I0909 13:49:16.860150 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0909 13:49:16.860234 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] }] } I0909 13:49:16.860327 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:16.860353 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0909 13:49:16.860452 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0909 13:49:16.860482 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0909 13:49:16.860577 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:16.860609 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0909 13:49:16.860690 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:16.860730 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] }] } I0909 13:49:16.860806 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.860875 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0909 13:49:16.860974 52398 executor.go:111] Tasks: 69 done / 116 total; 32 can run W0909 13:49:16.861061 52398 api.go:53] PageSize not implemented I0909 13:49:16.861077 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.861150 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.861145 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:16.861187 52398 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0909 13:49:16.861227 52398 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0909 13:49:16.861310 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:16.861251 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:16.861356 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:16.861378 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0909 13:49:16.861402 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:16.861417 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0909 13:49:16.861528 52398 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } I0909 13:49:16.861556 52398 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4"] } I0909 13:49:16.861582 52398 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } W0909 13:49:16.861644 52398 api.go:53] PageSize not implemented I0909 13:49:16.861662 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.861654 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.861716 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.861750 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:16.861825 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0909 13:49:16.861923 52398 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0909 13:49:16.861992 52398 api.go:53] PageSize not implemented I0909 13:49:16.861970 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0909 13:49:16.862037 52398 dnsname.go:118] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0909 13:49:16.862026 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.862029 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:16.862098 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0909 13:49:16.862155 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0909 13:49:16.862209 52398 api.go:53] PageSize not implemented I0909 13:49:16.862236 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.864647 52398 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0909 13:49:16.864708 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:16.864776 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.864883 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.864931 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.865034 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.866182 52398 executor.go:111] Tasks: 112 done / 116 total; 2 can run W0909 13:49:16.866292 52398 api.go:53] PageSize not implemented I0909 13:49:16.866309 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.866423 52398 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0909 13:49:16.866520 52398 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0909 13:49:16.868368 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0909 13:49:16.868407 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0909 13:49:16.868431 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:16.868452 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0909 13:49:16.868471 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0909 13:49:16.868491 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0909 13:49:16.868513 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0909 13:49:16.868535 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0909 13:49:16.868557 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0909 13:49:16.868584 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0909 13:49:16.868607 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:16.868627 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:16.868646 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:16.868668 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0909 13:49:16.868687 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0909 13:49:16.868710 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:16.868731 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:16.868753 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0909 13:49:16.868773 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0909 13:49:16.868794 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:16.868817 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:16.868837 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0909 13:49:16.868858 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:16.868877 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0909 13:49:16.868897 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:16.868920 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:16.868944 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0909 13:49:16.869773 52398 delete_cluster.go:128] Looking for cloud resources to delete W0909 13:49:16.869815 52398 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0909 13:49:16.869827 52398 keypairs.go:95] DescribeKeyPairs: { } I0909 13:49:16.869851 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.869889 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0909 13:49:16.869929 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.869966 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.869993 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.870019 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.870044 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.870071 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0909 13:49:16.870103 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.870138 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:16.870155 52398 natgateway.go:117] DescribeNatGateways: { } I0909 13:49:16.870180 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0909 13:49:16.870201 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0909 13:49:16.870228 52398 api.go:53] PageSize not implemented I0909 13:49:16.870238 52398 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0909 13:49:16.870260 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:16.870275 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:16.870284 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0909 13:49:16.870305 52398 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0909 13:49:16.870319 52398 targetgroups.go:35] PageSize not implemented I0909 13:49:16.870330 52398 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" ] } I0909 13:49:16.870366 52398 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0909 13:49:16.870390 52398 iamrole.go:89] ListRoles: { } I0909 13:49:16.870420 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:16.870436 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:16.870462 52398 internetgateways.go:101] DescribeInternetGateways: { } I0909 13:49:16.870487 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:16.870523 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0909 13:49:16.870547 52398 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0909 13:49:16.870567 52398 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0909 13:49:16.870887 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0909 13:49:16.870941 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0909 13:49:16.870936 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0909 13:49:16.870992 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0909 13:49:16.870991 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:nodes.private-shared-ip.example.com ok autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0909 13:49:16.870923 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-ip.example.com. ok I0909 13:49:16.871019 52398 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } I0909 13:49:16.871035 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1 ok volume:vol-1 ok I0909 13:49:16.871077 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } autoscaling-group:bastion.private-shared-ip.example.com ok I0909 13:49:16.871061 52398 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } iam-instance-profile:bastions.private-shared-ip.example.com ok load-balancer:api-private-shared-ip-exa-ohatqj ok I0909 13:49:16.871077 52398 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4 ok I0909 13:49:16.871101 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0909 13:49:16.871037 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0909 13:49:16.871164 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:16.871178 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-1 ok I0909 13:49:16.871145 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok I0909 13:49:16.871207 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0909 13:49:16.871249 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 } ] } W0909 13:49:16.871343 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:16.871353 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0909 13:49:16.871373 52398 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-3 ok I0909 13:49:16.871408 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:16.871438 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0909 13:49:16.871505 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:16.871514 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0909 13:49:16.871539 52398 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0909 13:49:16.871591 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0909 13:49:16.871625 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:16.871624 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0909 13:49:16.871651 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } I0909 13:49:16.871674 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } W0909 13:49:16.871712 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:16.871726 52398 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0909 13:49:16.871722 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0909 13:49:16.871739 52398 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0909 13:49:16.871756 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0909 13:49:16.871757 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0909 13:49:16.871776 52398 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0909 13:49:16.871793 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0909 13:49:16.871818 52398 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0909 13:49:16.871828 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0909 13:49:16.871844 52398 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0909 13:49:16.871866 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0909 13:49:16.871885 52398 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0909 13:49:16.871919 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0909 13:49:16.871965 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0909 13:49:16.872154 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.34s) === RUN TestLifecycleManyAddons I0909 13:49:16.872275 52398 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:16.872359 52398 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:16.872381 52398 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:16.872410 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:16.872435 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:16.872464 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:16.872488 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:16.872505 52398 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:16.872530 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:16.872547 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:16.872576 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0909 13:49:16.887351 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:16.889290 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.889341 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:16.959469 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:17.624105 52398 iamrole.go:89] ListRoles: { } I0909 13:49:17.625084 52398 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0909 13:49:17.625342 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:17.627176 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:17.627386 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:17.627553 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:17.627708 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:17.627926 52398 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.628247 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0909 13:49:17.628320 52398 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0909 13:49:17.628368 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:17.628423 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:17.628498 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:17.628674 52398 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.629029 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0909 13:49:17.629261 52398 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0909 13:49:17.629376 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:17.629356 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" }] } I0909 13:49:17.630628 52398 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0909 13:49:17.630619 52398 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.630906 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:17.631599 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:17.631885 52398 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0909 13:49:17.632111 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:17.632962 52398 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0909 13:49:17.633140 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:17.633130 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0909 13:49:17.633318 52398 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0909 13:49:17.633282 52398 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0909 13:49:17.633324 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:17.633354 52398 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0909 13:49:17.633403 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0909 13:49:17.633454 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:network-interface/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignPrivateIpAddresses\",\n \"ec2:AttachNetworkInterface\",\n \"ec2:CreateNetworkInterface\",\n \"ec2:DeleteNetworkInterface\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeTags\",\n \"ec2:DetachNetworkInterface\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ec2:UnassignPrivateIpAddresses\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0909 13:49:17.633547 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0909 13:49:17.633540 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:17.633602 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:17.633636 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:17.633664 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0909 13:49:17.633775 52398 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0909 13:49:17.633830 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:17.633874 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:17.633899 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:17.634012 52398 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0909 13:49:17.634062 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:17.634110 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:17.634139 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:17.634162 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0909 13:49:17.634216 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0909 13:49:17.634288 52398 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0909 13:49:17.634289 52398 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0909 13:49:17.634339 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:17.634367 52398 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] } I0909 13:49:17.634411 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:17.634515 52398 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0909 13:49:17.636025 52398 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:network-interface/*\"\n ]\n },\n {\n \"Action\": [\n \"acm:DescribeCertificate\",\n \"acm:ListCertificates\",\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignPrivateIpAddresses\",\n \"ec2:AttachNetworkInterface\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateNetworkInterface\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateTags\",\n \"ec2:DeleteNetworkInterface\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeInternetGateways\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSnapshots\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcPeeringConnections\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachNetworkInterface\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:UnassignPrivateIpAddresses\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListenerCertificates\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeRules\",\n \"elasticloadbalancing:DescribeTags\",\n \"elasticloadbalancing:DescribeTargetGroupAttributes\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteSnapshot\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddListenerCertificates\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteRule\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyRule\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:ModifyTargetGroupAttributes\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:RemoveListenerCertificates\",\n \"elasticloadbalancing:RemoveTags\",\n \"elasticloadbalancing:SetIpAddressType\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\",\n \"elasticloadbalancing:SetSecurityGroups\",\n \"elasticloadbalancing:SetSubnets\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateRule\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0909 13:49:17.637474 52398 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0909 13:49:17.637667 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.637815 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:17.637885 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:17.638070 52398 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0909 13:49:17.638221 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" }] }] } I0909 13:49:17.638393 52398 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0909 13:49:17.638447 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.638667 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0909 13:49:17.638878 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.639018 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:17.639115 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:17.639214 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" }] }] } I0909 13:49:17.639354 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.639490 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.639636 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0909 13:49:17.639789 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:17.639889 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:17.639965 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.640118 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0909 13:49:17.640164 52398 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0909 13:49:17.640202 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.640336 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:17.640493 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.640700 52398 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.640851 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" }] }] } I0909 13:49:17.641003 52398 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0909 13:49:17.642540 52398 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0909 13:49:17.642789 52398 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0909 13:49:17.643011 52398 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0909 13:49:17.643067 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:17.643104 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:17.643182 52398 dns.go:232] Pre-creating DNS records I0909 13:49:17.643213 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:17.649285 52398 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0909 13:49:17.652717 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:17.652836 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:17.723952 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:18.455490 52398 iamrole.go:89] ListRoles: { } I0909 13:49:18.456553 52398 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0909 13:49:18.456937 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.457123 52398 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0909 13:49:18.457204 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.457392 52398 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0909 13:49:18.457536 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.457644 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:18.457664 52398 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:18.457703 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:18.457731 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.460034 52398 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0909 13:49:18.460266 52398 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0909 13:49:18.460259 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:18.460389 52398 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0909 13:49:18.460380 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0909 13:49:18.460486 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:18.460545 52398 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:18.460672 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.460825 52398 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:18.460872 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.461026 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.461155 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0909 13:49:18.461309 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:18.465399 52398 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0909 13:49:18.465521 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:18.465683 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:18.465772 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:18.465878 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:18.466208 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0909 13:49:18.466350 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:18.466504 52398 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0909 13:49:18.467909 52398 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0909 13:49:18.468103 52398 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0909 13:49:18.468275 52398 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0909 13:49:18.471228 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0909 13:49:18.471305 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0909 13:49:18.471381 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0909 13:49:18.471449 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0909 13:49:18.471517 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0909 13:49:18.471583 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0909 13:49:18.471652 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0909 13:49:18.471723 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0909 13:49:18.471796 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0909 13:49:18.471858 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0909 13:49:18.471932 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0909 13:49:18.471983 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0909 13:49:18.472043 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0909 13:49:18.472106 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0909 13:49:18.472167 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:18.472229 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0909 13:49:18.472291 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:18.472354 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0909 13:49:18.472414 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0909 13:49:18.472472 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0909 13:49:18.472536 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0909 13:49:18.472600 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0909 13:49:18.472666 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:18.472725 52398 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0909 13:49:18.474407 52398 delete_cluster.go:128] Looking for cloud resources to delete W0909 13:49:18.474494 52398 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0909 13:49:18.474529 52398 keypairs.go:95] DescribeKeyPairs: { } I0909 13:49:18.474600 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.474679 52398 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0909 13:49:18.474764 52398 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.474847 52398 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.474916 52398 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.475002 52398 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.475062 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.475127 52398 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0909 13:49:18.475204 52398 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.475283 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0909 13:49:18.475343 52398 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0909 13:49:18.475415 52398 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0909 13:49:18.475477 52398 api.go:53] PageSize not implemented I0909 13:49:18.475510 52398 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0909 13:49:18.475554 52398 loadbalancers.go:35] PageSize not implemented I0909 13:49:18.475584 52398 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0909 13:49:18.475623 52398 targetgroups.go:35] PageSize not implemented I0909 13:49:18.475653 52398 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0909 13:49:18.475729 52398 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0909 13:49:18.475789 52398 iamrole.go:89] ListRoles: { } I0909 13:49:18.475853 52398 zones.go:93] ListHostedZonesPages { } I0909 13:49:18.475896 52398 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0909 13:49:18.475962 52398 internetgateways.go:101] DescribeInternetGateways: { } I0909 13:49:18.476030 52398 routetable.go:53] DescribeRouteTables: { } I0909 13:49:18.476088 52398 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.minimal.example.com ok autoscaling-config:lt-2 ok autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0909 13:49:18.476573 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0909 13:49:18.476626 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0909 13:49:18.476709 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0909 13:49:18.476752 52398 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0909 13:49:18.476802 52398 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0909 13:49:18.476682 52398 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.minimal.example.com. ok I0909 13:49:18.476673 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0909 13:49:18.476999 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:18.477038 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0909 13:49:18.477097 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0909 13:49:18.477200 52398 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0909 13:49:18.477283 52398 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0909 13:49:18.477423 52398 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0909 13:49:18.477456 52398 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0909 13:49:18.477511 52398 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok I0909 13:49:18.477616 52398 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0909 13:49:18.477700 52398 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0909 13:49:18.477788 52398 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0909 13:49:18.477885 52398 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0909 13:49:18.477909 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0909 13:49:18.477999 52398 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0909 13:49:18.478019 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0909 13:49:18.478056 52398 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0909 13:49:18.478113 52398 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0909 13:49:18.478199 52398 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0909 13:49:18.478216 52398 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0909 13:49:18.478249 52398 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0909 13:49:18.478321 52398 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0909 13:49:18.478398 52398 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0909 13:49:18.478467 52398 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0909 13:49:18.478726 52398 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleManyAddons (1.61s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 52.006s === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.005s === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0909 13:48:27.000308 52585 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.004s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.004s === RUN TestPodController I0909 13:48:27.033484 52587 pod.go:59] starting pod controller I0909 13:48:27.033714 52587 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.00s) I0909 13:48:27.033801 52587 pod.go:107] Got stop signal PASS I0909 13:48:27.033831 52587 pod.go:65] shutting down pod controller ok k8s.io/kops/dns-controller/pkg/watchers 0.039s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.004s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000035260} route53_test.go:93: Got zone list: [0xc000035290] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc0000940b8} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000353c0} route53_test.go:93: Got zone list: [0xc0000353f0] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc0000940d0} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc0000354f0} route53_test.go:170: Successfully added managed DNS zone: &{0xc000301e00 0xc0000940f0} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000356b0} route53_test.go:93: Got zone list: [0xc0000356e0] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc000094110} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000035840} route53_test.go:93: Got zone list: [0xc000035870] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc000094138} route53_test.go:187: Successfully added resource record set: {0xc0000fe600 0xc000094158} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000035b50} route53_test.go:93: Got zone list: [0xc000035b80] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc0000941e8} route53_test.go:199: Successfully added resource record set: {0xc0000fe780 0xc000094208} route53_test.go:118: Got 1 recordsets: [0xc000035d20] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000035eb0} route53_test.go:93: Got zone list: [0xc000035ee0] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc0000942a8} route53_test.go:221: Successfully added resource record set: {0xc0000fe900 0xc0000942c8} route53_test.go:228: Correctly failed to add duplicate resource record {0xc0000fe900 0xc0000942c8}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00032c2a0} route53_test.go:93: Got zone list: [0xc00032c2d0] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc000094388} route53_test.go:246: Successfully removed resource set {0xc0000feb00 0xc0000943a8} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00032c5b0} route53_test.go:93: Got zone list: [0xc00032c5e0] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc000094438} route53_test.go:264: Successfully removed resource set {0xc0000fec80 0xc000094458} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00032c8f0} route53_test.go:93: Got zone list: [0xc00032c920] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc0000944f0} commontests.go:192: Successfully added resource record set: {0xc0000fee00 0xc000094510} commontests.go:45: Correctly replaced resource record {0xc0000fee00 0xc000094510} -> {0xc0000fef00 0xc000094560} commontests.go:142: Got 1 recordsets: [0xc00032cc30] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00032ce60} route53_test.go:93: Got zone list: [0xc00032ce90] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc000094618} commontests.go:192: Successfully added resource record set: {0xc0000ff180 0xc000094638} commontests.go:70: Correctly replaced resource record {0xc0000ff180 0xc000094638} -> {0xc0000ff280 0xc000094688} commontests.go:142: Got 1 recordsets: [0xc00032d1a0] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00032d420} route53_test.go:93: Got zone list: [0xc00032d450] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc000094750} commontests.go:192: Successfully added resource record set: {0xc0000ff500 0xc000094770} commontests.go:142: Got 2 recordsets: [0xc00032d6c0 0xc00032d6e0] commontests.go:142: Got 2 recordsets: [0xc00032d750 0xc00032d770] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00032d970} route53_test.go:93: Got zone list: [0xc00032d9a0] route53_test.go:98: Got at least 1 zone in list:&{0xc000301bc0 0xc000094878} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.015s === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.005s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc0001140a0} clouddns_test.go:71: Got zone list: [0xc000382138] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc000382108} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc0001141a0} clouddns_test.go:71: Got zone list: [0xc0003821f8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc0003821c8} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc0001142a0} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc00033af40 0xc0003822b8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc000114360} clouddns_test.go:71: Got zone list: [0xc000382378] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc000382348} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc000114480} clouddns_test.go:71: Got zone list: [0xc0003824b0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc000382480} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc0001145c0} clouddns_test.go:71: Got zone list: [0xc000382618] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc0003825e8} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc000114720} clouddns_test.go:71: Got zone list: [0xc000382828] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc0003827f8} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc000114880} clouddns_test.go:71: Got zone list: [0xc0003829c0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc000382990} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc0001149c0} clouddns_test.go:71: Got zone list: [0xc000382b28] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc000382af8} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc000114b20} clouddns_test.go:71: Got zone list: [0xc000382cf0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc000382cc0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc000114c80} clouddns_test.go:71: Got zone list: [0xc000382fc0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc000382f90} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc000114e00} clouddns_test.go:71: Got zone list: [0xc0003832f0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc0003832c0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0005fc0d8 0xc000114fe0} clouddns_test.go:71: Got zone list: [0xc0003836c8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00033ad40 0xc000383698} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.063s === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.021s === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.045s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0909 13:48:34.712175 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.712366 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.712422 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.712488 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.712542 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.712617 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.712691 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.712720 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.712757 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.712793 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.721786 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I0909 13:48:34.725760 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.725855 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.725883 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.725913 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.725936 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.725976 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.726019 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.726042 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.726068 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.726085 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.727216 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.00s) === RUN TestContainerdBuilder_Simple I0909 13:48:34.730696 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.730788 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.730814 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.730843 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.730866 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.730905 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.730935 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.730954 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.730976 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.730993 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.732122 53147 zones.go:93] ListHostedZonesPages { } I0909 13:48:34.736555 53147 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0909 13:48:34.736587 53147 assetstore.go:111] bin/containerd I0909 13:48:34.736597 53147 assetstore.go:111] bin/containerd-shim I0909 13:48:34.736605 53147 assetstore.go:111] bin/containerd-shim-runc-v1 I0909 13:48:34.736613 53147 assetstore.go:111] bin/containerd-shim-runc-v2 I0909 13:48:34.736620 53147 assetstore.go:111] bin/containerd-stress I0909 13:48:34.736630 53147 assetstore.go:111] bin/ctr I0909 13:48:34.736709 53147 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0909 13:48:34.736725 53147 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0909 13:48:34.740273 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.740396 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.740439 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.740489 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.740550 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.740601 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.740647 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.740677 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.740717 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.740745 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.742105 53147 zones.go:93] ListHostedZonesPages { } I0909 13:48:34.743927 53147 containerd.go:60] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0909 13:48:34.744985 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.745095 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.745128 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.745172 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.745209 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.745253 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.745341 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.745374 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.745424 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.745449 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.746558 53147 zones.go:93] ListHostedZonesPages { } I0909 13:48:34.748159 53147 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0909 13:48:34.748916 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.749016 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.749046 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.749089 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.749122 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.749166 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.749225 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.749255 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.749295 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.749321 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.750402 53147 zones.go:93] ListHostedZonesPages { } I0909 13:48:34.751968 53147 assetstore.go:108] Matching assets for "^docker/": I0909 13:48:34.751991 53147 assetstore.go:111] docker/containerd I0909 13:48:34.752006 53147 assetstore.go:111] docker/containerd-shim I0909 13:48:34.752019 53147 assetstore.go:111] docker/ctr I0909 13:48:34.752032 53147 assetstore.go:111] docker/docker I0909 13:48:34.752044 53147 assetstore.go:111] docker/docker-init I0909 13:48:34.752056 53147 assetstore.go:111] docker/docker-proxy I0909 13:48:34.752070 53147 assetstore.go:111] docker/dockerd I0909 13:48:34.752082 53147 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0909 13:48:34.755502 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.755604 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.755638 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.755686 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.755720 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.755767 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.755830 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.755862 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.755901 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.755925 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.757220 53147 zones.go:93] ListHostedZonesPages { } I0909 13:48:34.758834 53147 assetstore.go:108] Matching assets for "^docker/": I0909 13:48:34.758858 53147 assetstore.go:111] docker/containerd I0909 13:48:34.758872 53147 assetstore.go:111] docker/containerd-shim I0909 13:48:34.758885 53147 assetstore.go:111] docker/ctr I0909 13:48:34.758897 53147 assetstore.go:111] docker/docker I0909 13:48:34.758910 53147 assetstore.go:111] docker/docker-init I0909 13:48:34.758921 53147 assetstore.go:111] docker/docker-proxy I0909 13:48:34.758933 53147 assetstore.go:111] docker/dockerd I0909 13:48:34.758946 53147 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0909 13:48:34.761782 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.761884 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.761921 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.761967 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.762001 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.762049 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.762115 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.762160 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.762204 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.762230 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.763304 53147 zones.go:93] ListHostedZonesPages { } I0909 13:48:34.765104 53147 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0909 13:48:34.765315 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.765408 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.765439 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.765480 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.765517 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.765559 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.765618 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.765649 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.765686 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.765710 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.766911 53147 zones.go:93] ListHostedZonesPages { } I0909 13:48:34.769288 53147 assetstore.go:108] Matching assets for "^docker/": I0909 13:48:34.769318 53147 assetstore.go:111] docker/docker I0909 13:48:34.769332 53147 assetstore.go:111] docker/docker-containerd I0909 13:48:34.769344 53147 assetstore.go:111] docker/docker-containerd-ctr I0909 13:48:34.769356 53147 assetstore.go:111] docker/docker-containerd-shim I0909 13:48:34.769368 53147 assetstore.go:111] docker/docker-init I0909 13:48:34.769380 53147 assetstore.go:111] docker/docker-proxy I0909 13:48:34.769403 53147 assetstore.go:111] docker/docker-runc I0909 13:48:34.769422 53147 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0909 13:48:34.779016 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.779136 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.779173 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.779217 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.779251 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.779291 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.779329 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.779356 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.779390 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.779415 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.783065 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.01s) === RUN TestDockerHooksBuilder I0909 13:48:34.786657 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.786787 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.786826 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.786872 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.786911 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.786960 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.787009 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.787038 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.787080 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.787106 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.790096 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.01s) === RUN TestKopsControllerBuilder I0909 13:48:34.793607 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.793723 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.793759 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.793801 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.793846 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.793892 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.793934 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.793960 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.793993 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.794016 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.796808 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0909 13:48:34.806439 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.806573 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.806615 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.806661 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.806696 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.806743 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.806826 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.806858 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.806891 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.806914 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.809720 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.02s) === RUN TestDeddicatedAPIServerBuilder I0909 13:48:34.823057 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.823172 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.823208 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.823249 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.823281 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.823344 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.823383 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.823406 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.823438 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.823461 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.829933 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.02s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0909 13:48:34.841450 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.841586 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.841633 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.841689 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.841733 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.841788 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.841834 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.841865 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.841907 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.841934 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.844745 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0909 13:48:34.855817 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.855958 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.856005 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.856057 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.856104 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.856158 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.856205 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.856237 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.856281 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.856310 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.859311 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.02s) === RUN TestKubeAPIServerBuilderAMD64 I0909 13:48:34.876999 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:34.877126 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:34.877190 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:34.877280 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:34.877323 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:34.877383 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:34.877439 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:34.877482 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:34.877537 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:34.877564 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:34.881563 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.12s) === RUN TestKubeAPIServerBuilderARM64 I0909 13:48:36.001036 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.001199 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.001275 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.001355 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.001423 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.001504 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.001579 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.001640 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.001709 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.001765 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.004551 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.13s) === RUN TestKubeControllerManagerBuilder I0909 13:48:36.134028 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.134167 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.134204 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.134259 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.134297 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.134346 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.134387 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.134416 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.134454 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.134479 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.136339 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0909 13:48:36.144997 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.145094 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.145123 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.145153 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.145178 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.145211 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.145239 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.145257 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.145281 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.145299 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.147085 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.15s) === RUN TestKubeControllerManagerBuilderARM64 I0909 13:48:36.292808 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.292931 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.292973 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.293027 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.293069 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.293129 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.293172 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.293204 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.293242 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.293269 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.296051 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.14s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0909 13:48:36.428607 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.428775 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.428844 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.428927 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.428972 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.429023 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.429065 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.429097 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.429132 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.429158 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.431213 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0909 13:48:36.437441 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.437577 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.437630 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.437682 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.437724 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.437774 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.437821 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.437852 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.437892 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.437920 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.441068 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.14s) === RUN TestKubeProxyBuilderARM64 I0909 13:48:36.575480 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.575613 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.575655 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.575710 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.575755 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.575808 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.575852 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.575884 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.575927 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.575961 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.584834 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.13s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0909 13:48:36.708732 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.708821 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.708850 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.708879 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.708903 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.708937 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.708965 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.708982 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.709005 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.709021 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.710710 53147 zones.go:93] ListHostedZonesPages { } W0909 13:48:36.713048 53147 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0909 13:48:36.714710 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.714800 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.714826 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.714851 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.714876 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.714905 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.714931 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.714948 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.714969 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.714986 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.716653 53147 zones.go:93] ListHostedZonesPages { } W0909 13:48:36.842258 53147 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.13s) === RUN TestKubeSchedulerBuilderARM64 I0909 13:48:36.846589 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.846721 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.846762 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.846809 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.846850 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.846905 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.846951 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.846987 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.847025 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.847054 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.850054 53147 zones.go:93] ListHostedZonesPages { } W0909 13:48:36.974395 53147 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.13s) === RUN TestKubectlBuilder I0909 13:48:36.976202 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.976293 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.976320 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.976351 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.976376 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.976410 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.976436 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.976453 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.976476 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.976493 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.978278 53147 zones.go:93] ListHostedZonesPages { } I0909 13:48:36.980006 53147 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0909 13:48:36.982216 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.982321 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.982379 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.982424 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.982462 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.982513 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.982554 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.982584 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.982611 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.982629 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.984147 53147 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0909 13:48:36.987663 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.987737 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.987776 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.987806 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.987832 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.987863 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.987889 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.987906 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.987931 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.987947 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.989233 53147 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0909 13:48:36.992481 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.992568 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.992593 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.992623 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.992646 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.992677 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.992702 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.992718 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.992741 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.992758 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:36.994547 53147 zones.go:93] ListHostedZonesPages { } I0909 13:48:36.996151 53147 assetstore.go:108] Matching assets for "protokube$": I0909 13:48:36.996170 53147 assetstore.go:111] /opt/kops/bin/protokube I0909 13:48:36.996189 53147 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0909 13:48:36.996217 53147 assetstore.go:108] Matching assets for "channels$": I0909 13:48:36.996227 53147 assetstore.go:111] /opt/kops/bin/channels I0909 13:48:36.996234 53147 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I0909 13:48:36.999279 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:36.999367 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:36.999393 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:36.999424 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:36.999449 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:36.999480 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:36.999506 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:36.999523 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:36.999545 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:36.999561 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:37.001374 53147 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0909 13:48:37.003677 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:37.003766 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:37.003791 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:37.003820 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:37.003858 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:37.003891 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:37.003917 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:37.003934 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:37.003958 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:37.003974 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:37.005683 53147 zones.go:93] ListHostedZonesPages { } I0909 13:48:37.007312 53147 update_service.go:91] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0909 13:48:37.007821 53147 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:48:37.007907 53147 internetgateways.go:66] CreateInternetGateway: { } I0909 13:48:37.007933 53147 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:48:37.007964 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:48:37.007988 53147 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:48:37.008018 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:48:37.008046 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:48:37.008062 53147 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:48:37.008086 53147 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:48:37.008102 53147 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:48:37.009802 53147 zones.go:93] ListHostedZonesPages { } I0909 13:48:37.011438 53147 update_service.go:86] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 2.382s === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.026s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.038s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.027s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.040s === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0909 13:48:34.646906 53152 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0909 13:48:34.647071 53152 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.017s === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0909 13:48:34.697546 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.697639 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.697686 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.697719 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.697756 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0909 13:48:34.697967 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698004 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698034 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698068 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698105 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698137 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698175 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698204 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698240 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698279 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698309 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698347 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698395 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698426 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698457 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698501 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698529 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698558 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0909 13:48:34.698587 53153 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0909 13:48:34.698781 53153 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0909 13:48:34.698855 53153 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.075s === RUN TestManagedFieldsMigrator === RUN TestManagedFieldsMigrator/csa-migration === RUN TestManagedFieldsMigrator/merge-owners === RUN TestManagedFieldsMigrator/ssa-unchanged === RUN TestManagedFieldsMigrator/unknown-unchanged --- PASS: TestManagedFieldsMigrator (0.00s) --- PASS: TestManagedFieldsMigrator/csa-migration (0.00s) --- PASS: TestManagedFieldsMigrator/merge-owners (0.00s) --- PASS: TestManagedFieldsMigrator/ssa-unchanged (0.00s) --- PASS: TestManagedFieldsMigrator/unknown-unchanged (0.00s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.024s === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0909 13:48:34.761291 53157 builder.go:230] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.09s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0909 13:48:34.762319 53157 builder.go:230] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0909 13:48:34.763352 53157 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0909 13:48:34.764523 53157 builder.go:230] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0909 13:48:34.765852 53157 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0909 13:48:34.767089 53157 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0909 13:48:34.768042 53157 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0909 13:48:34.768923 53157 copyfile.go:208] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0909 13:48:34.768945 53157 copyfile.go:209] S3, Google Cloud Storage, and File Paths are supported. E0909 13:48:34.768957 53157 copyfile.go:210] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0909 13:48:34.768969 53157 copyfile.go:211] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.134s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.003s === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.058s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.021s === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.00s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.067s === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.024s === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.004s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.003s === RUN TestHasExtraFields === RUN TestHasExtraFields/minimal === RUN TestHasExtraFields/extraFields === RUN TestHasExtraFields/spec2 === RUN TestHasExtraFields/isolateMasters === RUN TestHasExtraFields/instanceGroup --- PASS: TestHasExtraFields (0.02s) --- PASS: TestHasExtraFields/minimal (0.01s) --- PASS: TestHasExtraFields/extraFields (0.00s) --- PASS: TestHasExtraFields/spec2 (0.00s) --- PASS: TestHasExtraFields/isolateMasters (0.00s) --- PASS: TestHasExtraFields/instanceGroup (0.00s) PASS ok k8s.io/kops/pkg/edit 0.037s === RUN TestFlagToFalse I0909 13:48:41.798918 54517 featureflag_test.go:32] Created flag Unittest1 I0909 13:48:41.798962 54517 featureflag.go:166] FeatureFlag "UnitTest1"=false I0909 13:48:41.798973 54517 featureflag.go:166] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0909 13:48:41.799041 54517 featureflag.go:166] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) === RUN TestGetPositive --- PASS: TestGetPositive (0.00s) === RUN TestGetNegative --- PASS: TestGetNegative (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.003s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.028s === RUN TestRenderInstanceGroupZones W0909 13:48:41.818009 54519 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.023s === RUN TestWarmPoolOnlyRoll I0909 13:48:54.278672 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.278766 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.278795 54996 instancegroups.go:136] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0909 13:48:54.279402 54996 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0909 13:48:54.279448 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.280643 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.280687 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.285836 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.285890 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.285917 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.285934 54996 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0909 13:48:54.288543 54996 instancegroups.go:431] Draining the node: "master-1a.local". I0909 13:48:54.289414 54996 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0909 13:48:54.289541 54996 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0909 13:48:54.289578 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.290723 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.290764 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.295986 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.296060 54996 instancegroups.go:431] Draining the node: "master-1b.local". I0909 13:48:54.297009 54996 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0909 13:48:54.297082 54996 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0909 13:48:54.297116 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.298251 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.298282 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.303494 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.303541 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.303563 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.303588 54996 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0909 13:48:54.304690 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.305517 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.305552 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.306695 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.306716 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.311933 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.312014 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:54.312865 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:54.312899 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.314029 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.314049 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.319250 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.319319 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:54.320388 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:54.320426 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.321586 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.321610 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.326837 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.326879 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.326910 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.326923 54996 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0909 13:48:54.327901 54996 instancegroups.go:431] Draining the node: "node-2a.local". I0909 13:48:54.328608 54996 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0909 13:48:54.328642 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.329804 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.329840 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.335048 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.335111 54996 instancegroups.go:431] Draining the node: "node-2b.local". I0909 13:48:54.336034 54996 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0909 13:48:54.336067 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.337214 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.337238 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.342470 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.342526 54996 instancegroups.go:431] Draining the node: "node-2c.local". I0909 13:48:54.343235 54996 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0909 13:48:54.343265 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.344403 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.344436 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.349698 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.349742 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0909 13:48:54.350174 54996 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0909 13:48:54.350198 54996 instancegroups.go:467] waiting for 1ms after terminating instance W0909 13:48:54.351345 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0909 13:48:54.351401 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0909 13:48:54.351446 54996 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0909 13:48:54.351464 54996 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0909 13:48:54.351492 54996 instancegroups.go:467] waiting for 1ms after terminating instance W0909 13:48:54.352614 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0909 13:48:54.352678 54996 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0909 13:48:54.352694 54996 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0909 13:48:54.352718 54996 instancegroups.go:467] waiting for 1ms after terminating instance W0909 13:48:54.353851 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0909 13:48:54.353889 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0909 13:48:54.353939 54996 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0909 13:48:54.353957 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.353984 54996 instancegroups.go:467] waiting for 1ms after terminating instance W0909 13:48:54.355115 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0909 13:48:54.355157 54996 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0909 13:48:54.355184 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:54.355207 54996 instancegroups.go:467] waiting for 1ms after terminating instance W0909 13:48:54.356326 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0909 13:48:54.356369 54996 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0909 13:48:54.356395 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:54.356417 54996 instancegroups.go:467] waiting for 1ms after terminating instance W0909 13:48:54.357528 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0909 13:48:54.357549 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0909 13:48:54.357589 54996 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0909 13:48:54.357615 54996 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0909 13:48:54.357639 54996 instancegroups.go:467] waiting for 1ms after terminating instance W0909 13:48:54.358770 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0909 13:48:54.358811 54996 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0909 13:48:54.358827 54996 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0909 13:48:54.358854 54996 instancegroups.go:467] waiting for 1ms after terminating instance W0909 13:48:54.359974 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0909 13:48:54.360019 54996 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0909 13:48:54.360041 54996 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0909 13:48:54.360070 54996 instancegroups.go:467] waiting for 1ms after terminating instance W0909 13:48:54.361190 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0909 13:48:54.361220 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0909 13:48:54.361564 54996 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0909 13:48:54.361595 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.362735 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.362772 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.362794 54996 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.362838 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.362862 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.362880 54996 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.362894 54996 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0909 13:48:54.363788 54996 instancegroups.go:431] Draining the node: "master-1a.local". I0909 13:48:54.365141 54996 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0909 13:48:54.365217 54996 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0909 13:48:54.365248 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.366389 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.366425 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.366458 54996 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.366507 54996 instancegroups.go:431] Draining the node: "master-1b.local". I0909 13:48:54.367538 54996 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0909 13:48:54.367604 54996 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0909 13:48:54.367639 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.368770 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.368803 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.368824 54996 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.368846 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.368870 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.368891 54996 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.368909 54996 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0909 13:48:54.369896 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.370769 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.370804 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.371944 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.371990 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.372011 54996 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.372061 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:54.372907 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:54.372938 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.374058 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.374086 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.374104 54996 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.374134 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:54.374839 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:54.374875 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.375995 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.376024 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.376045 54996 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.376062 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.376083 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.376100 54996 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.376117 54996 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0909 13:48:54.377160 54996 instancegroups.go:431] Draining the node: "node-2a.local". I0909 13:48:54.378172 54996 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0909 13:48:54.378220 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.379356 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.379389 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.379411 54996 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.379450 54996 instancegroups.go:431] Draining the node: "node-2b.local". I0909 13:48:54.380207 54996 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0909 13:48:54.380239 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.381439 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.381492 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.381516 54996 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.381565 54996 instancegroups.go:431] Draining the node: "node-2c.local". I0909 13:48:54.382396 54996 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0909 13:48:54.382427 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.383571 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.383600 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0909 13:48:54.383620 54996 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0909 13:48:54.383643 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0909 13:48:54.384052 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0909 13:48:54.384337 54996 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0909 13:48:54.384367 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.385550 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.385599 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.390869 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.390946 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.390974 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.390993 54996 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0909 13:48:54.391902 54996 instancegroups.go:431] Draining the node: "master-1a.local". I0909 13:48:54.392673 54996 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0909 13:48:54.392745 54996 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0909 13:48:54.392782 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.393952 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.393992 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.399444 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.399522 54996 instancegroups.go:431] Draining the node: "master-1b.local". I0909 13:48:54.400568 54996 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0909 13:48:54.400652 54996 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0909 13:48:54.400690 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.401833 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.401863 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.407087 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.407147 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.407188 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.407223 54996 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0909 13:48:54.408503 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.409395 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.409476 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.410659 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.410692 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.415940 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.416005 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:54.416902 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:54.416936 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.418077 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.418101 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.423326 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.423395 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:54.424333 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:54.424368 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.425538 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.425573 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.430788 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.430829 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.430850 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.430865 54996 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0909 13:48:54.431852 54996 instancegroups.go:431] Draining the node: "node-2a.local". I0909 13:48:54.432575 54996 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0909 13:48:54.432609 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.433747 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.433776 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.438989 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.439058 54996 instancegroups.go:431] Draining the node: "node-2b.local". I0909 13:48:54.440010 54996 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0909 13:48:54.440045 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.441213 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.441243 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.446474 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.446542 54996 instancegroups.go:431] Draining the node: "node-2c.local". I0909 13:48:54.447441 54996 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0909 13:48:54.447479 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.448636 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.448671 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.454537 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.454581 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0909 13:48:54.454739 54996 rollingupdate.go:111] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0909 13:48:54.455052 54996 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0909 13:48:54.455073 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.456200 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.456230 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0909 13:48:54.456244 54996 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0909 13:48:54.456455 54996 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0909 13:48:54.456472 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.457608 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.457639 54996 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0909 13:48:54.457660 54996 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0909 13:48:54.457889 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.457918 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0909 13:48:54.457936 54996 instancegroups.go:506] Cluster did not validate within 0s E0909 13:48:54.457953 54996 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0909 13:48:54.458116 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.458139 54996 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0909 13:48:54.458157 54996 instancegroups.go:506] Cluster did not validate within 0s E0909 13:48:54.458174 54996 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0909 13:48:54.458395 54996 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0909 13:48:54.458422 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.459555 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.459592 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.464826 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.464901 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.464925 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.464940 54996 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0909 13:48:54.465720 54996 instancegroups.go:431] Draining the node: "master-1a.local". I0909 13:48:54.466379 54996 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0909 13:48:54.466448 54996 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0909 13:48:54.466476 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.467619 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.467649 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0909 13:48:54.467662 54996 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0909 13:48:54.467921 54996 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0909 13:48:54.467938 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.469056 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.469081 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.474359 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.474448 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.474478 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.474496 54996 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0909 13:48:54.475356 54996 instancegroups.go:431] Draining the node: "master-1a.local". I0909 13:48:54.476079 54996 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0909 13:48:54.476152 54996 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0909 13:48:54.476181 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.477322 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.477367 54996 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0909 13:48:54.477390 54996 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0909 13:48:54.477688 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.477709 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.477720 54996 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0909 13:48:54.478601 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.479297 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.479326 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.480449 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.480472 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.485683 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.485740 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:54.486327 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:54.486352 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.487467 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.487490 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.492715 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.492784 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:54.493720 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:54.493754 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.494890 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.494920 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.500142 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.500197 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0909 13:48:54.500563 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.500591 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0909 13:48:54.500610 54996 instancegroups.go:506] Cluster did not validate within 0s E0909 13:48:54.500624 54996 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0909 13:48:54.500860 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.500886 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0909 13:48:54.500900 54996 instancegroups.go:506] Cluster did not validate within 0s E0909 13:48:54.500913 54996 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0909 13:48:54.501133 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.501159 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0909 13:48:54.501174 54996 instancegroups.go:506] Cluster did not validate within 0s E0909 13:48:54.501187 54996 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupExitableError I0909 13:48:54.501409 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.501427 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.501442 54996 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0909 13:48:54.502468 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.503178 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.503209 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.504372 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.504417 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.509636 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.509703 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:54.510332 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:54.510360 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.511477 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.511497 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.516717 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.516769 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:54.517538 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:54.517571 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.518706 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.518735 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.523910 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.523959 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.523991 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0909 13:48:54.524017 54996 instancegroups.go:506] Cluster did not validate within 0s E0909 13:48:54.524039 54996 rollingupdate.go:218] failed to roll InstanceGroup "node-2": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateValidationErrorInstanceGroupExitableError (0.02s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0909 13:48:54.526508 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.526550 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.526568 54996 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0909 13:48:54.527635 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.528363 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.528394 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.529538 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.529584 54996 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0909 13:48:54.529603 54996 instancegroups.go:506] Cluster did not validate within 0s E0909 13:48:54.529618 54996 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0909 13:48:54.529817 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.529835 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.529845 54996 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0909 13:48:54.530553 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.531097 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.531122 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.532255 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.532296 54996 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0909 13:48:54.532323 54996 instancegroups.go:506] Cluster did not validate within 0s E0909 13:48:54.532360 54996 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0909 13:48:54.532610 54996 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0909 13:48:54.532643 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.533963 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.534004 54996 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0909 13:48:54.535454 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.540717 54996 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0909 13:48:54.541873 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.547100 54996 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0909 13:48:54.548285 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.553501 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.553596 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.553629 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.553649 54996 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0909 13:48:54.554580 54996 instancegroups.go:431] Draining the node: "master-1a.local". I0909 13:48:54.555366 54996 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0909 13:48:54.555440 54996 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0909 13:48:54.555471 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.556624 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.556654 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.562773 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.562860 54996 instancegroups.go:431] Draining the node: "master-1b.local". I0909 13:48:54.563687 54996 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0909 13:48:54.563755 54996 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0909 13:48:54.563784 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.564950 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.565014 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.570243 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.570289 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.570309 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.570321 54996 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0909 13:48:54.571039 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.571543 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.571566 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.572661 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.572695 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.577920 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.577976 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:54.578569 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:54.578593 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.579751 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.579800 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.584967 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.585017 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:54.585615 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:54.585638 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.586755 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.586794 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.591969 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.592015 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.592044 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.592061 54996 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0909 13:48:54.592973 54996 instancegroups.go:431] Draining the node: "node-2a.local". I0909 13:48:54.593529 54996 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0909 13:48:54.593552 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.594671 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.594700 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.600142 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.600198 54996 instancegroups.go:431] Draining the node: "node-2b.local". I0909 13:48:54.600841 54996 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0909 13:48:54.600866 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.601997 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.602042 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.607294 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.607367 54996 instancegroups.go:431] Draining the node: "node-2c.local". I0909 13:48:54.608088 54996 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0909 13:48:54.608123 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.609250 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.609295 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.614502 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.614542 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0909 13:48:54.614793 54996 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0909 13:48:54.614812 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.615929 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.615966 54996 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0909 13:48:54.617117 54996 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0909 13:48:54.618239 54996 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0909 13:48:54.619345 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.624557 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.624651 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.624677 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.624694 54996 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0909 13:48:54.625484 54996 instancegroups.go:431] Draining the node: "master-1a.local". I0909 13:48:54.626187 54996 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0909 13:48:54.626254 54996 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0909 13:48:54.626284 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.627384 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.627413 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.632599 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.632664 54996 instancegroups.go:431] Draining the node: "master-1b.local". I0909 13:48:54.633520 54996 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0909 13:48:54.633587 54996 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0909 13:48:54.633618 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.634747 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.634773 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.639971 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.640028 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.640049 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.640066 54996 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0909 13:48:54.641116 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.641858 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.641893 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.643021 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.643049 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.648346 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.648417 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:54.649275 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:54.649313 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.650445 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.650470 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.655658 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.655721 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:54.656624 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:54.656658 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.657798 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.657819 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.662998 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.663027 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.663039 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.663049 54996 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0909 13:48:54.663703 54996 instancegroups.go:431] Draining the node: "node-2a.local". I0909 13:48:54.664198 54996 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0909 13:48:54.664223 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.665363 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.665383 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.670558 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.670628 54996 instancegroups.go:431] Draining the node: "node-2b.local". I0909 13:48:54.671313 54996 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0909 13:48:54.671339 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.672484 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.672506 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.677681 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.677723 54996 instancegroups.go:431] Draining the node: "node-2c.local". I0909 13:48:54.678295 54996 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0909 13:48:54.678318 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.679458 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.679483 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.684664 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.684694 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0909 13:48:54.685144 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.685160 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.685168 54996 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0909 13:48:54.685636 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.686106 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.686127 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.687266 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.687287 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.692479 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.692531 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:54.693135 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:54.693159 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.694272 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.694302 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.700233 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.700284 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0909 13:48:54.700617 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.700644 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.700663 54996 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0909 13:48:54.701450 54996 instancegroups.go:431] Draining the node: "master-1a.local". I0909 13:48:54.702173 54996 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0909 13:48:54.702238 54996 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0909 13:48:54.702268 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.703415 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.703436 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.708630 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.708686 54996 instancegroups.go:431] Draining the node: "master-1b.local". I0909 13:48:54.709513 54996 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0909 13:48:54.709574 54996 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0909 13:48:54.709604 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.710750 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.710771 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.715972 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.716011 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0909 13:48:54.716382 54996 instancegroups.go:216] Rolling updates for InstanceGroup bastion-1 are disabled W0909 13:48:54.716426 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0909 13:48:54.716449 54996 instancegroups.go:216] Rolling updates for InstanceGroup master-1 are disabled W0909 13:48:54.716467 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0909 13:48:54.716488 54996 instancegroups.go:216] Rolling updates for InstanceGroup node-1 are disabled W0909 13:48:54.716501 54996 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0909 13:48:54.716520 54996 instancegroups.go:216] Rolling updates for InstanceGroup node-2 are disabled I0909 13:48:54.716535 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0909 13:48:54.716787 54996 instancegroups.go:604] Detaching instance "bastion-1a", in group "bastion-1". I0909 13:48:54.716838 54996 instancegroups.go:203] waiting for 1ms after detaching instance I0909 13:48:54.717980 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.718002 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.723207 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.723245 54996 instancegroups.go:216] Rolling updates for InstanceGroup bastion-1 are disabled I0909 13:48:54.723302 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.723324 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.723338 54996 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0909 13:48:54.724099 54996 instancegroups.go:216] Rolling updates for InstanceGroup master-1 are disabled I0909 13:48:54.724125 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.724143 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.724155 54996 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0909 13:48:54.725023 54996 instancegroups.go:602] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0909 13:48:54.725084 54996 instancegroups.go:203] waiting for 1ms after detaching instance I0909 13:48:54.726218 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.726241 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.731424 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.731452 54996 instancegroups.go:216] Rolling updates for InstanceGroup node-1 are disabled I0909 13:48:54.731467 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.731478 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.731488 54996 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0909 13:48:54.732135 54996 instancegroups.go:602] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0909 13:48:54.732177 54996 instancegroups.go:203] waiting for 1ms after detaching instance I0909 13:48:54.733285 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.733304 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:54.738487 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.738528 54996 instancegroups.go:216] Rolling updates for InstanceGroup node-2 are disabled I0909 13:48:54.738548 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0909 13:48:54.738793 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.738817 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.738826 54996 instancegroups.go:342] Tainting 7 nodes in "node-1" instancegroup. I0909 13:48:54.740440 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.740994 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.741034 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.742172 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.742223 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.742264 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:54.742286 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:54.743373 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:54.743613 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:54.743651 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.744929 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.744990 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.845347 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.845441 54996 instancegroups.go:431] Draining the node: "node-1e.local". I0909 13:48:54.845468 54996 instancegroups.go:431] Draining the node: "node-1d.local". I0909 13:48:54.846444 54996 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0909 13:48:54.846702 54996 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0909 13:48:54.846747 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.847880 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.847920 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.847956 54996 instancegroups.go:431] Draining the node: "node-1f.local". I0909 13:48:54.848682 54996 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0909 13:48:54.848719 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.849849 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.849907 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.952468 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.952546 54996 instancegroups.go:431] Draining the node: "node-1g.local". I0909 13:48:54.953287 54996 instancegroups.go:625] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0909 13:48:54.953332 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.954514 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.954566 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.954583 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0909 13:48:54.954795 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.954817 54996 instancegroups.go:537] Cluster validated. I0909 13:48:54.954830 54996 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0909 13:48:54.956279 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:54.956309 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:54.957264 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:54.957493 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:54.957530 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:54.958684 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:54.958738 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.059256 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.059341 54996 instancegroups.go:431] Draining the node: "node-1d.local". I0909 13:48:55.059395 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:55.060438 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:55.060723 54996 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0909 13:48:55.060771 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.061931 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.061959 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.061981 54996 instancegroups.go:431] Draining the node: "node-1e.local". I0909 13:48:55.062457 54996 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0909 13:48:55.062483 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.063621 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.063665 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.164114 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.164186 54996 instancegroups.go:431] Draining the node: "node-1f.local". I0909 13:48:55.164812 54996 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0909 13:48:55.164848 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.165981 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.166018 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.166040 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0909 13:48:55.166279 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.166308 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.166322 54996 instancegroups.go:342] Tainting 7 nodes in "master-1" instancegroup. I0909 13:48:55.168465 54996 instancegroups.go:431] Draining the node: "master-1a.local". I0909 13:48:55.169168 54996 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0909 13:48:55.169246 54996 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0909 13:48:55.169292 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.170467 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.170512 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.170544 54996 instancegroups.go:431] Draining the node: "master-1c.local". I0909 13:48:55.170616 54996 instancegroups.go:431] Draining the node: "master-1b.local". I0909 13:48:55.171210 54996 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0909 13:48:55.171263 54996 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0909 13:48:55.171399 54996 instancegroups.go:688] skipping deregistration of instance "master-1c", as part of instancegroup with role "ControlPlane" I0909 13:48:55.171428 54996 instancegroups.go:625] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0909 13:48:55.171450 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.172585 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.172652 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.273372 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.273451 54996 instancegroups.go:431] Draining the node: "master-1e.local". I0909 13:48:55.273512 54996 instancegroups.go:431] Draining the node: "master-1d.local". I0909 13:48:55.274603 54996 instancegroups.go:688] skipping deregistration of instance "master-1d", as part of instancegroup with role "ControlPlane" I0909 13:48:55.274686 54996 instancegroups.go:625] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0909 13:48:55.274891 54996 instancegroups.go:688] skipping deregistration of instance "master-1e", as part of instancegroup with role "ControlPlane" I0909 13:48:55.274958 54996 instancegroups.go:625] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0909 13:48:55.275001 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.276137 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.276175 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.276210 54996 instancegroups.go:431] Draining the node: "master-1f.local". I0909 13:48:55.276974 54996 instancegroups.go:688] skipping deregistration of instance "master-1f", as part of instancegroup with role "ControlPlane" I0909 13:48:55.277025 54996 instancegroups.go:625] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0909 13:48:55.277061 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.278195 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.278253 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.378669 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.378743 54996 instancegroups.go:431] Draining the node: "master-1g.local". I0909 13:48:55.379710 54996 instancegroups.go:688] skipping deregistration of instance "master-1g", as part of instancegroup with role "ControlPlane" I0909 13:48:55.379778 54996 instancegroups.go:625] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0909 13:48:55.379823 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.381000 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.381049 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.381064 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.21s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0909 13:48:55.381301 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.381323 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.381331 54996 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0909 13:48:55.382397 54996 instancegroups.go:602] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0909 13:48:55.382441 54996 instancegroups.go:203] waiting for 1ms after detaching instance I0909 13:48:55.383581 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.383631 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.383650 54996 instancegroups.go:602] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0909 13:48:55.383717 54996 instancegroups.go:203] waiting for 1ms after detaching instance I0909 13:48:55.384853 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.384896 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.384935 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:55.384972 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:55.385939 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:55.386203 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:55.386246 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.387375 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.387429 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.487846 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.487904 54996 instancegroups.go:431] Draining the node: "node-1d.local". I0909 13:48:55.487953 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:55.488793 54996 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0909 13:48:55.488946 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:55.488973 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.490115 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.490142 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.490169 54996 instancegroups.go:431] Draining the node: "node-1e.local". I0909 13:48:55.490662 54996 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0909 13:48:55.490691 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.491834 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.491869 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.592284 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.592359 54996 instancegroups.go:431] Draining the node: "node-1f.local". I0909 13:48:55.593214 54996 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0909 13:48:55.593255 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.594388 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.594426 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.594447 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0909 13:48:55.594693 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.594724 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.594738 54996 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0909 13:48:55.598837 54996 instancegroups.go:602] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0909 13:48:55.598911 54996 instancegroups.go:602] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0909 13:48:55.598960 54996 instancegroups.go:203] waiting for 1ms after detaching instance I0909 13:48:55.600134 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.600180 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.600225 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:55.600272 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:55.600832 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:55.601184 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:55.601212 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.602336 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.602377 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.702760 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.702843 54996 instancegroups.go:431] Draining the node: "node-1d.local". I0909 13:48:55.702879 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:55.704027 54996 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0909 13:48:55.704281 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:55.704329 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.705489 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.705517 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.705538 54996 instancegroups.go:431] Draining the node: "node-1e.local". I0909 13:48:55.706002 54996 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0909 13:48:55.706034 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.707158 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.707226 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.807890 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.807955 54996 instancegroups.go:431] Draining the node: "node-1f.local". I0909 13:48:55.808572 54996 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0909 13:48:55.808612 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.809760 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.809788 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.809805 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0909 13:48:55.809983 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.809997 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.810004 54996 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0909 13:48:55.810415 54996 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0909 13:48:55.810440 54996 instancegroups.go:602] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0909 13:48:55.810462 54996 instancegroups.go:203] waiting for 1ms after detaching instance I0909 13:48:55.811607 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.811626 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:55.816798 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.816834 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:55.816845 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:55.817585 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:55.817607 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.817744 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:55.817762 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.818771 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.818805 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:55.824029 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.824055 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.01s) === RUN TestRollingUpdateDetachFails I0909 13:48:55.824226 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.824240 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.824247 54996 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0909 13:48:55.824661 54996 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0909 13:48:55.824697 54996 instancegroups.go:191] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0909 13:48:55.824705 54996 instancegroups.go:602] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0909 13:48:55.824729 54996 instancegroups.go:191] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0909 13:48:55.824747 54996 instancegroups.go:203] waiting for 1ms after detaching instance I0909 13:48:55.825864 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.825884 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:55.830990 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.831028 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:55.831082 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:55.831807 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:55.831832 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.831975 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:55.831996 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.832951 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.832978 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:55.838233 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.838262 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.01s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0909 13:48:55.838516 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.838550 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.838567 54996 instancegroups.go:342] Tainting 4 nodes in "node-1" instancegroup. I0909 13:48:55.839432 54996 instancegroups.go:602] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0909 13:48:55.839473 54996 instancegroups.go:203] waiting for 1ms after detaching instance I0909 13:48:55.840586 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.840616 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.840631 54996 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0909 13:48:55.840681 54996 instancegroups.go:203] waiting for 1ms after detaching instance I0909 13:48:55.841805 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.841831 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.841860 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:55.841881 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:55.841971 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:55.842744 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:55.842800 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.843208 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:55.843238 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.843374 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:55.843399 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.843929 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.944570 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.944645 54996 instancegroups.go:431] Draining the node: "node-1d.local". I0909 13:48:55.945600 54996 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0909 13:48:55.945648 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.946799 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.946827 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.946841 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0909 13:48:55.947003 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.947022 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.947031 54996 instancegroups.go:342] Tainting 7 nodes in "node-1" instancegroup. I0909 13:48:55.948228 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:55.948656 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:55.948682 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.949852 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.949880 54996 instancegroups.go:537] Cluster validated. I0909 13:48:55.949905 54996 instancegroups.go:431] Draining the node: "node-1e.local". I0909 13:48:55.949973 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:55.950594 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:55.950754 54996 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0909 13:48:55.950780 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:55.951930 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:55.951969 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:56.052601 54996 instancegroups.go:537] Cluster validated. I0909 13:48:56.052699 54996 instancegroups.go:431] Draining the node: "node-1g.local". I0909 13:48:56.052718 54996 instancegroups.go:431] Draining the node: "node-1f.local". I0909 13:48:56.053907 54996 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0909 13:48:56.054142 54996 instancegroups.go:625] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0909 13:48:56.054183 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:56.055353 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:56.055414 54996 instancegroups.go:537] Cluster validated. I0909 13:48:56.055461 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:56.056136 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:56.056166 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:56.057320 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:56.057372 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:56.158153 54996 instancegroups.go:537] Cluster validated. I0909 13:48:56.158214 54996 instancegroups.go:431] Draining the node: "node-1d.local". I0909 13:48:56.158911 54996 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0909 13:48:56.158945 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:56.160120 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:56.160167 54996 instancegroups.go:537] Cluster validated. I0909 13:48:56.160184 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.21s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0909 13:48:56.160398 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:56.160413 54996 instancegroups.go:537] Cluster validated. I0909 13:48:56.160422 54996 instancegroups.go:136] deleting warm pool instance "node-1-wp-a" I0909 13:48:56.160436 54996 instancegroups.go:136] deleting warm pool instance "node-1-wp-b" I0909 13:48:56.160445 54996 instancegroups.go:136] deleting warm pool instance "node-1-wp-c" I0909 13:48:56.160459 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0909 13:48:56.160574 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:56.160590 54996 instancegroups.go:537] Cluster validated. I0909 13:48:56.160597 54996 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0909 13:48:56.161200 54996 instancegroups.go:136] deleting warm pool instance "node-1-wp-a" I0909 13:48:56.161220 54996 instancegroups.go:136] deleting warm pool instance "node-1-wp-b" I0909 13:48:56.161230 54996 instancegroups.go:136] deleting warm pool instance "node-1-wp-c" I0909 13:48:56.161256 54996 instancegroups.go:431] Draining the node: "node-1a.local". I0909 13:48:56.161880 54996 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0909 13:48:56.161922 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:56.163426 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:56.163452 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:56.168639 54996 instancegroups.go:537] Cluster validated. I0909 13:48:56.168683 54996 instancegroups.go:431] Draining the node: "node-1b.local". I0909 13:48:56.169232 54996 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0909 13:48:56.169261 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:56.170407 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:56.170425 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:56.175810 54996 instancegroups.go:537] Cluster validated. I0909 13:48:56.175847 54996 instancegroups.go:431] Draining the node: "node-1c.local". I0909 13:48:56.176348 54996 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0909 13:48:56.176375 54996 instancegroups.go:467] waiting for 1ms after terminating instance I0909 13:48:56.177511 54996 instancegroups.go:501] Validating the cluster. I0909 13:48:56.177531 54996 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0909 13:48:56.182721 54996 instancegroups.go:537] Cluster validated. I0909 13:48:56.182753 54996 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.998s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.011s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.014s === RUN TestParse E0909 13:48:54.201616 54998 versions.go:96] unable to parse Kubernetes version "" E0909 13:48:54.201722 54998 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.011s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.034s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_with_admin_and_internal_option === RUN TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_without_admin_and_with_internal_option --- PASS: TestBuildKubecfg (0.22s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.01s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_with_admin_and_internal_option (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_without_admin_and_with_internal_option (0.00s) PASS ok k8s.io/kops/pkg/kubeconfig 0.288s === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.02s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.094s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/ControlPlane === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/ControlPlane (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.069s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.084s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0909 13:48:54.258734 55005 featureflag.go:166] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.070s === RUN Test_RunEtcdManagerBuilder I0909 13:48:54.266938 55006 featureflag.go:166] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0909 13:48:54.313989 55006 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 3 W0909 13:48:54.315354 55006 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.05s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.03s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.130s === RUN Test_RunKubeApiserverBuilder I0909 13:48:54.260066 55007 featureflag.go:166] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.02s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.02s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.090s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.02s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.090s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.026s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.056s === RUN TestServerGroupBuilder I0909 13:49:02.012610 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.027699 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.039534 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.064724 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.077066 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.166183 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.185243 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.205097 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.221800 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.232203 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.245258 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.255934 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.262840 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.268909 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.304914 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.313168 55854 featureflag.go:166] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.31s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.399s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.027s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.014s === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.026s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (2.13s) --- PASS: TestIssueCert/ca (0.95s) --- PASS: TestIssueCert/client (0.86s) --- PASS: TestIssueCert/clientOneYear (0.32s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 2.145s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.008s === RUN TestAddUntaggedRouteTables I0909 13:49:02.007379 55862 routetable.go:53] DescribeRouteTables: { } W0909 13:49:02.007520 55862 tags.go:67] cluster tag not found on route-table:rtb-1234 I0909 13:49:02.007538 55862 aws.go:229] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I0909 13:49:02.007729 55862 iaminstanceprofile.go:193] ListInstanceProfiles: { } W0909 13:49:02.007821 55862 aws.go:2059] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I0909 13:49:02.007944 55862 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I0909 13:49:02.008079 55862 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0909 13:49:02.008188 55862 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0909 13:49:02.008317 55862 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0909 13:49:02.008445 55862 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0909 13:49:02.008499 55862 volumes.go:128] DescribeVolumes: { } W0909 13:49:02.008543 55862 aws.go:2212] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0909 13:49:02.008764 55862 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:02.008879 55862 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:02.008923 55862 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:02.009009 55862 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:02.009060 55862 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:02.009130 55862 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:02.009185 55862 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:02.009219 55862 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:02.009275 55862 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:02.009325 55862 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:02.009359 55862 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0909 13:49:02.009405 55862 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0909 13:49:02.009442 55862 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0909 13:49:02.009506 55862 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0909 13:49:02.009543 55862 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0909 13:49:02.009591 55862 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0909 13:49:02.009641 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0909 13:49:02.009681 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0909 13:49:02.009723 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0909 13:49:02.009760 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0909 13:49:02.009801 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0909 13:49:02.009837 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0909 13:49:02.009880 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0909 13:49:02.009914 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0909 13:49:02.009954 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0909 13:49:02.009992 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0909 13:49:02.010034 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0909 13:49:02.010068 55862 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.081s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.028s === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.072s === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.006s === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.004s === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.004s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.004s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.004s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.039s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.099s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.005s === RUN TestRemovesDuplicateGuardedBlocks W0909 13:49:09.249160 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.249232 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.249245 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.249260 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.249272 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.249285 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.249296 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.249309 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.249322 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.249334 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.249345 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.249358 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.249372 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.249385 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.249396 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.249410 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.267822 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.267852 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.267926 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.267938 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.267994 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268004 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268061 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268072 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268140 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268152 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268230 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268243 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268301 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268312 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268365 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268376 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268466 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268478 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268533 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268544 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268595 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268606 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268670 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268681 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268754 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268766 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268825 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268836 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268894 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268904 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.268957 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.268968 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269025 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269035 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269110 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269122 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269178 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269197 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269299 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269311 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269365 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269376 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269432 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269444 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269498 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269508 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269559 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269570 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269625 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269637 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269692 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269708 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269765 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269776 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269829 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269839 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269892 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269903 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.269956 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.269966 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.270018 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.270029 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.270094 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.270106 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.270169 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.270181 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.270243 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.270255 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.270310 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.270321 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.270372 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.270383 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.270438 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.270452 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.270535 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.270549 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.270608 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.270619 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.270680 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.270691 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.270745 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.270757 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.271331 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.271353 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.271464 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.271481 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.271565 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.271582 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.271662 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.271678 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.271755 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.271770 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.271859 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.271876 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.271960 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.271976 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.272075 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.272093 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.272173 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.272190 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.272266 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.272283 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.272416 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.272434 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.272513 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.272529 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.272609 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.272628 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.272707 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.272724 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.272806 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.272823 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.272903 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.272920 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.272997 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.273013 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.273093 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.273110 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.273187 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.273510 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.273602 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.273619 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.273695 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.273711 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.273787 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.273804 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.273931 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.273950 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.274048 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.274065 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.274144 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.274160 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.274238 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.274254 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.274333 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.274350 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.274436 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.274453 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.274531 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.274547 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.274623 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.274639 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.274715 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.274731 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.274813 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.274844 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.274926 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.274943 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.275022 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.275039 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.275120 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.275136 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.275215 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.275231 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.275314 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.275331 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.275407 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.275422 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.275502 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.275516 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.275596 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.275612 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.275690 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.275706 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.275786 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.275802 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.275881 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.275897 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.276007 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.276025 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.276104 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.276123 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.276200 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.276217 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.276293 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.276309 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.276402 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.276420 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.276500 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.276517 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.276596 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.276613 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.276691 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.276707 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.276790 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.276807 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.276884 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.276901 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.276977 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.276993 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.277076 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.277093 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.277193 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.277212 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.277291 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.277307 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.277384 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.277400 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.03s) === RUN TestRecoversFromBadNesting W0909 13:49:09.277826 57168 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0909 13:49:09.277841 57168 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0909 13:49:09.277854 57168 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0909 13:49:09.277865 57168 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0909 13:49:09.277877 57168 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0909 13:49:09.277887 57168 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0909 13:49:09.277897 57168 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0909 13:49:09.277927 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.277940 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.277953 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.277964 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.277976 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.277987 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.277998 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.278014 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.278026 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.278036 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.278047 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.278060 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.278072 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.278084 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.278095 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.278106 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.279959 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.279988 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.280082 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.280105 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.280190 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.280209 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.280302 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.280322 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.280407 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.280424 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.280522 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.280541 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.280668 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.280685 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.280769 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.280793 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.280874 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.280889 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.280972 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.280988 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.281073 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.281089 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.281170 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.281185 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.281286 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.281303 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.281386 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.281403 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.281481 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.281501 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.281579 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.281598 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.281682 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.281699 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.281779 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.281796 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.281880 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.281897 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.281978 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.281996 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.282080 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.282107 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.282191 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.282207 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.282286 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.282302 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.282384 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.282402 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.283122 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.283140 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.283224 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.283241 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.283320 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.283337 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.283417 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.283434 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.283520 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.283536 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.283616 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.283634 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.283741 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.283758 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.283845 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.283864 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.283943 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.283961 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.284037 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.284053 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.284127 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.284144 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.284223 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.284240 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.284321 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.284337 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.284415 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.284431 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.284509 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.284526 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.284605 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.284621 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.284698 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.284728 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.284823 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.284840 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.284921 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.284937 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.285011 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.285028 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.285109 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.285125 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.285202 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.285217 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.285292 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.285310 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.285390 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.285407 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.285485 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.285501 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.285581 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.285597 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.285677 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.285693 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.285772 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.285789 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.285889 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.285947 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.286035 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.286052 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.286134 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.286151 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.286260 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.286282 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.286360 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.286376 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.286452 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.286468 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.286543 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.286559 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.286641 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.286657 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.286739 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.286755 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.286831 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.286847 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.286924 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.286940 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.287016 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.287033 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.287110 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.287129 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.287220 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.287238 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.287336 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.287353 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.287433 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.287449 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.287536 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.287552 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.287629 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.287646 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.287721 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.287738 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.287821 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.287837 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.287914 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.287930 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.288009 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.288025 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.288103 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.288119 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.288199 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.288215 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.288299 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.288314 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.288390 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.288405 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.288479 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.288495 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.288598 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.288618 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.288699 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.288716 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.288799 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.288815 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.288896 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.288914 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.288993 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.289009 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.289093 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.289109 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.289186 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.289203 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.289280 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.289297 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.289377 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.289394 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.289473 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.289491 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.289576 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.289593 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.289672 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.289689 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.289772 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.289789 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.289872 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.289888 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.289963 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.289979 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.290058 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.290075 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.290151 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.290168 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.290245 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.290261 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.290337 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.290354 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0909 13:49:09.290431 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0909 13:49:09.290447 57168 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.01s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.046s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.01s) PASS ok k8s.io/kops/tests/codecs 0.042s === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.02s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.054s === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.02s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.02s) PASS ok k8s.io/kops/tests/integration/conversion 0.043s === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0909 13:49:09.271832 57176 files.go:57] Writing file "/tmp/TestWriteFile2222704501/001/SubDir/test1.tmp" I0909 13:49:09.271982 57176 files.go:113] Changing file mode for "/tmp/TestWriteFile2222704501/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.01s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.038s === RUN TestBootstrapChannelBuilder_BuildTasks I0909 13:49:12.298453 57877 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:12.298606 57877 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:12.298640 57877 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:12.298677 57877 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:12.298708 57877 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:12.298754 57877 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:12.298790 57877 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:12.298808 57877 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:12.298832 57877 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:12.298856 57877 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:12.308846 57877 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.363972 57877 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.527284 57877 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.603934 57877 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.689639 57877 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.778978 57877 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.837815 57877 zones.go:93] ListHostedZonesPages { } I0909 13:49:12.895237 57877 zones.go:93] ListHostedZonesPages { } I0909 13:49:13.029639 57877 zones.go:93] ListHostedZonesPages { } I0909 13:49:13.576585 57877 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.32s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0909 13:49:13.623196 57877 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:13.623276 57877 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:13.623323 57877 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:13.623358 57877 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:13.623385 57877 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:13.623417 57877 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:13.623445 57877 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:13.623462 57877 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:13.623484 57877 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:13.623501 57877 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:13.623520 57877 featureflag.go:169] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0909 13:49:13.624659 57877 zones.go:93] ListHostedZonesPages { } I0909 13:49:13.733495 57877 featureflag.go:169] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.11s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0909 13:49:13.733629 57877 featureflag.go:166] FeatureFlag "ImageDigest"=false I0909 13:49:13.733700 57877 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:13.733746 57877 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0909 13:49:13.733780 57877 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0909 13:49:13.733809 57877 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0909 13:49:13.733853 57877 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0909 13:49:13.733884 57877 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0909 13:49:13.733901 57877 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0909 13:49:13.733925 57877 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0909 13:49:13.733944 57877 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0909 13:49:13.737290 57877 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.11s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.24s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.10s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0909 13:49:14.206179 57877 new_cluster.go:1132] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestDefaultImage --- PASS: TestDefaultImage (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0909 13:49:14.242576 57877 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0909 13:49:14.242601 57877 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0909 13:49:14.242612 57877 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0909 13:49:14.242630 57877 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0909 13:49:14.242642 57877 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0909 13:49:14.242662 57877 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0909 13:49:14.242682 57877 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0909 13:49:14.242701 57877 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0909 13:49:14.242714 57877 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0909 13:49:14.242733 57877 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0909 13:49:14.242745 57877 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0909 13:49:14.242764 57877 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0909 13:49:14.242790 57877 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0909 13:49:14.242810 57877 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0909 13:49:14.242821 57877 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0909 13:49:14.242832 57877 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0909 13:49:14.242844 57877 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0909 13:49:14.242855 57877 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0909 13:49:14.242867 57877 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0909 13:49:14.242878 57877 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0909 13:49:14.242889 57877 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0909 13:49:14.244079 57877 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN TestKopsFeatureEnabled === RUN TestKopsFeatureEnabled/Missing_feature === RUN TestKopsFeatureEnabled/Existing_feature I0909 13:49:14.244262 57877 featureflag.go:166] FeatureFlag "Scaleway"=true --- PASS: TestKopsFeatureEnabled (0.00s) --- PASS: TestKopsFeatureEnabled/Missing_feature (0.00s) --- PASS: TestKopsFeatureEnabled/Existing_feature (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 2.025s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0909 13:49:12.284151 57878 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2030733680/001 I0909 13:49:12.284454 57878 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2030733680/001 I0909 13:49:12.285110 57878 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2030733680/001 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestTGsARNsChunks --- PASS: TestTGsARNsChunks (0.01s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0909 13:49:12.293556 57878 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0909 13:49:12.293673 57878 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0909 13:49:12.293779 57878 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0909 13:49:12.293953 57878 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0909 13:49:12.294189 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.294277 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.294309 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.294560 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.294611 57878 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0909 13:49:12.294665 57878 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0909 13:49:12.294748 57878 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0909 13:49:12.294860 57878 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0909 13:49:12.294940 57878 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0909 13:49:12.294996 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.295053 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.295078 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.295127 57878 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0909 13:49:12.295196 57878 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0909 13:49:12.295279 57878 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0909 13:49:12.295436 57878 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0909 13:49:12.295513 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.295644 57878 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0909 13:49:12.295700 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.295737 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.295771 57878 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0909 13:49:12.295938 57878 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0909 13:49:12.296169 57878 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0909 13:49:12.296253 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.296294 57878 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.296335 57878 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0909 13:49:12.296427 57878 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.296629 57878 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0909 13:49:12.296711 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.296748 57878 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0909 13:49:12.296802 57878 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0909 13:49:12.296922 57878 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0909 13:49:12.296980 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.297047 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.297072 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.297109 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.297142 57878 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0909 13:49:12.297210 57878 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0909 13:49:12.297294 57878 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.297330 57878 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0909 13:49:12.297385 57878 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.297460 57878 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0909 13:49:12.297512 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0909 13:49:12.297564 57878 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0909 13:49:12.297660 57878 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0909 13:49:12.297719 57878 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0909 13:49:12.297752 57878 internetgateways.go:66] CreateInternetGateway: { } I0909 13:49:12.297775 57878 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0909 13:49:12.297848 57878 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0909 13:49:12.297904 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.297960 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.297985 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.298079 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.298118 57878 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0909 13:49:12.298164 57878 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0909 13:49:12.298244 57878 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0909 13:49:12.298346 57878 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0909 13:49:12.298418 57878 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0909 13:49:12.298470 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.298529 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.298554 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.298601 57878 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0909 13:49:12.298657 57878 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0909 13:49:12.298734 57878 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateTerraformRender I0909 13:49:12.299364 57878 target.go:116] Terraform output is in /tmp/TestLaunchTemplateTerraformRender185932427/001 I0909 13:49:12.299650 57878 target.go:116] Terraform output is in /tmp/TestLaunchTemplateTerraformRender185932427/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0909 13:49:12.300123 57878 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0909 13:49:12.300185 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.300310 57878 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0909 13:49:12.300368 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.300401 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.300433 57878 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0909 13:49:12.300541 57878 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0909 13:49:12.300806 57878 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0909 13:49:12.300860 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0909 13:49:12.300923 57878 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0909 13:49:12.301001 57878 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0909 13:49:12.301050 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.301102 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.301124 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.301159 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.301190 57878 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0909 13:49:12.301240 57878 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0909 13:49:12.301328 57878 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0909 13:49:12.301590 57878 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0909 13:49:12.301639 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.302397 57878 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0909 13:49:12.302471 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.302507 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.302545 57878 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0909 13:49:12.302613 57878 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0909 13:49:12.302761 57878 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0909 13:49:12.302821 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.302853 57878 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.302887 57878 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0909 13:49:12.302979 57878 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0909 13:49:12.303036 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.303098 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.303122 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.303158 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.303190 57878 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0909 13:49:12.303241 57878 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0909 13:49:12.303317 57878 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.303352 57878 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0909 13:49:12.303507 57878 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0909 13:49:12.303574 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.303687 57878 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0909 13:49:12.303737 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.303770 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.303803 57878 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0909 13:49:12.303891 57878 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0909 13:49:12.304005 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.304025 57878 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0909 13:49:12.304188 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.304222 57878 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.304256 57878 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0909 13:49:12.304305 57878 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0909 13:49:12.304400 57878 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0909 13:49:12.304456 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.304518 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.304542 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.304578 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.304611 57878 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0909 13:49:12.304670 57878 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0909 13:49:12.304753 57878 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.304781 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.304848 57878 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0909 13:49:12.304993 57878 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0909 13:49:12.305048 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.305159 57878 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0909 13:49:12.305213 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.305244 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.305280 57878 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0909 13:49:12.305336 57878 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0909 13:49:12.305468 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.305563 57878 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0909 13:49:12.305568 57878 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0909 13:49:12.305632 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.305666 57878 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.305699 57878 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0909 13:49:12.305803 57878 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0909 13:49:12.305862 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.305922 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.305947 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.305983 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.306020 57878 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0909 13:49:12.306071 57878 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0909 13:49:12.306183 57878 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.306209 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.306256 57878 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0909 13:49:12.306329 57878 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0909 13:49:12.306387 57878 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0909 13:49:12.306457 57878 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0909 13:49:12.306495 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.306532 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.306548 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.306615 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.306641 57878 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0909 13:49:12.306671 57878 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0909 13:49:12.306706 57878 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.306798 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0909 13:49:12.306823 57878 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0909 13:49:12.306850 57878 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0909 13:49:12.306919 57878 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0909 13:49:12.306958 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.306994 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.307010 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.307040 57878 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0909 13:49:12.307073 57878 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0909 13:49:12.307123 57878 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0909 13:49:12.307214 57878 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:12.307265 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.307344 57878 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0909 13:49:12.307378 57878 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0909 13:49:12.307399 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.307423 57878 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0909 13:49:12.307462 57878 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:12.307497 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0909 13:49:12.307535 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.307549 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.307571 57878 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0909 13:49:12.307593 57878 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0909 13:49:12.307689 57878 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0909 13:49:12.307751 57878 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:12.307787 57878 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0909 13:49:12.307824 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0909 13:49:12.307839 57878 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0909 13:49:12.307869 57878 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.072s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.042s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.022s === RUN TestDiskRenderAzure I0909 13:49:12.376914 57986 disk.go:112] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0909 13:49:12.377403 57986 disk.go:112] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0909 13:49:12.377706 57986 loadbalancer.go:132] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:124: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0909 13:49:12.378232 57986 loadbalancer.go:132] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0909 13:49:12.378537 57986 publicipaddress.go:109] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:102: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0909 13:49:12.378906 57986 publicipaddress.go:109] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0909 13:49:12.379188 57986 resourcegroup.go:108] Creating a new Resource Group with name: rg I0909 13:49:12.379201 57986 resourcegroup.go:110] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0909 13:49:12.379440 57986 resourcegroup.go:108] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0909 13:49:12.380097 57986 subnet.go:107] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0909 13:49:12.380377 57986 virtualnetwork.go:122] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0909 13:49:12.380652 57986 virtualnetwork.go:122] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0909 13:49:12.381018 57986 vmscaleset.go:272] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0909 13:49:12.381309 57986 vmscaleset.go:272] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.045s === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.040s === RUN TestProjectIAMBinding I0909 13:49:14.666394 58793 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:14.666984 58793 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0909 13:49:14.667008 58793 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:14.667510 58793 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0909 13:49:14.667536 58793 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:14.667681 58793 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0909 13:49:14.667783 58793 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:14.667891 58793 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0909 13:49:14.667914 58793 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:14.668037 58793 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0909 13:49:14.668062 58793 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:14.668105 58793 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0909 13:49:14.668198 58793 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:14.668453 58793 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0909 13:49:14.668475 58793 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:14.668766 58793 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0909 13:49:14.668792 58793 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0909 13:49:14.668916 58793 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.060s === RUN Test_OpenstackCloud_MakeCloud === RUN Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_false_when_not_configured === RUN Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_configured_value --- PASS: Test_OpenstackCloud_MakeCloud (0.00s) --- PASS: Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_false_when_not_configured (0.00s) --- PASS: Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_configured_value (0.00s) === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.02s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) === RUN Test_BuildClients === RUN Test_BuildClients/Empty_openstack_spec_means_no_load_balancer === RUN Test_BuildClients/When_octavia_is_set,_but_no_router,_an_error_should_be_returned === RUN Test_BuildClients/When_octavia_is_set,_and_there_is_a_router,_a_load-balancer_should_be_returned === RUN Test_BuildClients/When_octavia_is_not_set,_network_should_be_returned --- PASS: Test_BuildClients (0.00s) --- PASS: Test_BuildClients/Empty_openstack_spec_means_no_load_balancer (0.00s) --- PASS: Test_BuildClients/When_octavia_is_set,_but_no_router,_an_error_should_be_returned (0.00s) --- PASS: Test_BuildClients/When_octavia_is_set,_and_there_is_a_router,_a_load-balancer_should_be_returned (0.00s) --- PASS: Test_BuildClients/When_octavia_is_not_set,_network_should_be_returned (0.00s) === RUN Test_WaitForStatusActiveIsSuccessful --- PASS: Test_WaitForStatusActiveIsSuccessful (1.00s) === RUN Test_WaitForStatusActiveResultsInInstanceNotFound --- PASS: Test_WaitForStatusActiveResultsInInstanceNotFound (1.00s) === RUN Test_WaitForStatusActiveResultsInUnableToCreateServer --- PASS: Test_WaitForStatusActiveResultsInUnableToCreateServer (1.00s) === RUN Test_WaitForStatusActiveResultsInTimeout --- PASS: Test_WaitForStatusActiveResultsInTimeout (1.00s) === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 4.056s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.048s === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.024s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.005s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.026s === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0909 13:49:17.129648 59075 bindmount.go:223] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0909 13:49:17.129745 59075 bindmount.go:232] running mount command [mount -o remount,exec /containerized_mounter] I0909 13:49:17.129767 59075 bindmount.go:223] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0909 13:49:17.129802 59075 bindmount.go:243] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0909 13:49:17.129823 59075 bindmount.go:223] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0909 13:49:17.129845 59075 bindmount.go:223] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0909 13:49:17.129865 59075 bindmount.go:223] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0909 13:49:17.130644 59075 service.go:110] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.075s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.004s === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.005s === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.004s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.005s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.004s === RUN TestGetProxyEnvVars W0909 13:49:17.079680 59081 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.021s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.006s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.003s === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.003s === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.019s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W0909 13:49:20.444653 59167 s3context.go:235] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.072s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 137.3 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.26.3-r2.apk >>> kops: Build complete at Sat, 09 Sep 2023 13:49:25 +0000 elapsed time 0h 3m 30s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/5) Purging .makedepends-kops (20230909.134556) (2/5) Purging kubernetes (1.28.1-r1) (3/5) Purging go (1.21.1-r0) (4/5) Purging bash (5.2.15-r5) Executing bash-5.2.15-r5.pre-deinstall (5/5) Purging readline (8.2.1-r2) Executing busybox-1.36.1-r6.trigger OK: 379 MiB in 110 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...