I'm new to kubernetes and I'm having trouble to setup a functioning k8s cluster using EC2 AWS instances as nodes.
My master node seems fine and also my worker node (I currently only have this to for testing pourpose), but when i dig into it I see that all my pods in the worker node are crashing and getting the CrashLoopBackOff status.
Currently I got 4 pods in it; two for my hello-world api, one for flannel and one for kube-proxy (all crashing)
The status of both my nodes are 'ready'.
This is my flannel pod log, which I think is the root problem:
$ kubectl logs pod/kube-flannel-ds-hd79r --namespace=kube-flannel
Defaulted container "kube-flannel" out of: kube-flannel, install-cni-plugin (init), install-cni (init)
I0620 13:37:59.782900 1 main.go:211] CLI flags config: {etcdEndpoints:http://127.0.0.1:4001,http://127.0.0.1:2379 etcdPrefix:/coreos.com/network etcdKeyfile: etcdCertfile: etcdCAFile: etcdUsername: etcdPassword: version:false kubeSubnetMgr:true kubeApiUrl: kubeAnnotationPrefix:flannel.alpha.coreos.com kubeConfigFile: iface:[] ifaceRegex:[] ipMasq:true ifaceCanReach: subnetFile:/run/flannel/subnet.env publicIP: publicIPv6: subnetLeaseRenewMargin:60 healthzIP:0.0.0.0 healthzPort:0 iptablesResyncSeconds:5 iptablesForwardRules:true netConfPath:/etc/kube-flannel/net-conf.json setNodeNetworkUnavailable:true}
W0620 13:37:59.783087 1 client_config.go:618] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0620 13:37:59.796290 1 kube.go:139] Waiting 10m0s for node controller to sync
I0620 13:37:59.796318 1 kube.go:469] Starting kube subnet manager
I0620 13:37:59.801225 1 kube.go:490] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.244.0.0/24]
I0620 13:37:59.801277 1 kube.go:490] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.244.1.0/24]
I0620 13:38:00.796552 1 kube.go:146] Node controller sync successful
I0620 13:38:00.796592 1 main.go:231] Created subnet manager: Kubernetes Subnet Manager - worker01
I0620 13:38:00.796600 1 main.go:234] Installing signal handlers
I0620 13:38:00.796935 1 main.go:452] Found network config - Backend type: vxlan
I0620 13:38:00.800638 1 kube.go:669] List of node(worker01) annotations: map[string]string{"flannel.alpha.coreos.com/backend-data":"{\"VNI\":1,\"VtepMAC\":\"16:75:d6:62:b7:58\"}", "flannel.alpha.coreos.com/backend-type":"vxlan", "flannel.alpha.coreos.com/kube-subnet-manager":"true", "flannel.alpha.coreos.com/public-ip":"10.0.18.248", "kubeadm.alpha.kubernetes.io/cri-socket":"unix:///var/run/containerd/containerd.sock", "node.alpha.kubernetes.io/ttl":"0", "volumes.kubernetes.io/controller-managed-attach-detach":"true"}
I0620 13:38:00.800686 1 match.go:210] Determining IP address of default interface
I0620 13:38:00.801214 1 match.go:263] Using interface with name enX0 and address 10.0.18.248
I0620 13:38:00.801242 1 match.go:285] Defaulting external address to interface address (10.0.18.248)
I0620 13:38:00.801356 1 vxlan.go:141] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false
I0620 13:38:00.804033 1 kube.go:636] List of node(worker01) annotations: map[string]string{"flannel.alpha.coreos.com/backend-data":"{\"VNI\":1,\"VtepMAC\":\"16:75:d6:62:b7:58\"}", "flannel.alpha.coreos.com/backend-type":"vxlan", "flannel.alpha.coreos.com/kube-subnet-manager":"true", "flannel.alpha.coreos.com/public-ip":"10.0.18.248", "kubeadm.alpha.kubernetes.io/cri-socket":"unix:///var/run/containerd/containerd.sock", "node.alpha.kubernetes.io/ttl":"0", "volumes.kubernetes.io/controller-managed-attach-detach":"true"}
I0620 13:38:00.804098 1 vxlan.go:155] Interface flannel.1 mac address set to: 16:75:d6:62:b7:58
I0620 13:38:00.804634 1 iptables.go:51] Starting flannel in iptables mode...
W0620 13:38:00.804898 1 main.go:540] no subnet found for key: FLANNEL_IPV6_NETWORK in file: /run/flannel/subnet.env
W0620 13:38:00.804957 1 main.go:540] no subnet found for key: FLANNEL_IPV6_SUBNET in file: /run/flannel/subnet.env
I0620 13:38:00.805075 1 iptables.go:125] Setting up masking rules
I0620 13:38:00.814285 1 iptables.go:226] Changing default FORWARD chain policy to ACCEPT
I0620 13:38:00.817671 1 main.go:396] Wrote subnet file to /run/flannel/subnet.env
I0620 13:38:00.817689 1 main.go:400] Running backend.
I0620 13:38:00.831922 1 vxlan_network.go:65] watching for new subnet leases
I0620 13:38:00.832157 1 subnet.go:152] Batch elem [0] is { lease.Event{Type:0, Lease:lease.Lease{EnableIPv4:true, EnableIPv6:false, Subnet:ip.IP4Net{IP:0xaf40000, PrefixLen:0x18}, IPv6Subnet:ip.IP6Net{IP:(*ip.IP6)(nil), PrefixLen:0x0}, Attrs:lease.LeaseAttrs{PublicIP:0xa00150b, PublicIPv6:(*ip.IP6)(nil), BackendType:"vxlan", BackendData:json.RawMessage{0x7b, 0x22, 0x56, 0x4e, 0x49, 0x22, 0x3a, 0x31, 0x2c, 0x22, 0x56, 0x74, 0x65, 0x70, 0x4d, 0x41, 0x43, 0x22, 0x3a, 0x22, 0x33, 0x65, 0x3a, 0x31, 0x62, 0x3a, 0x30, 0x32, 0x3a, 0x38, 0x39, 0x3a, 0x64, 0x62, 0x3a, 0x35, 0x35, 0x22, 0x7d}, BackendV6Data:json.RawMessage(nil)}, Expiration:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Asof:0}} }
I0620 13:38:00.832479 1 vxlan_network.go:100] Received Subnet Event with VxLan: BackendType: vxlan, PublicIP: 10.0.21.11, PublicIPv6: (nil), BackendData: {"VNI":1,"VtepMAC":"3e:1b:02:89:db:55"}, BackendV6Data: (nil)
I0620 13:38:00.833388 1 iptables.go:372] bootstrap done
I0620 13:38:00.834596 1 main.go:421] Waiting for all goroutines to exit
I0620 13:38:00.840778 1 iptables.go:372] bootstrap done
I have already tried reseting and restarting my cluster, seeing if there is a memory problem in my node or with docker but nothing seems wrong with any of that.
Does anyone have any ideas?
