Questions tagged [etcd]
A highly-available key value store for shared configuration and service discovery
79
questions
13
votes
1
answer
2k
views
Graceful shutdown of Kubernetes cluster
We have a 4 node Kubernetes cluster, installed using Juju 2.0 and the official canonical Kubernetes charm. The sole purpose of this very local and private installation is development and testing.
One ...
- 131
5
votes
1
answer
3k
views
Why would using MongoDB in lieu of etcd as a key value store be a good or bad idea? [closed]
I'd like to get general thoughts from the community as to why MongoDB for key/value would OR would not be a suitable replacement for etcd or other purpose-built key/value store system (consol, ...
- 230
4
votes
2
answers
2k
views
All kube-system pods keep crashing, etcd receives sigterm
I'm trying to set up an on-premises Kubernetes cluster. For starters I used kubeadm to initialize my control plane on one node. But all kube-system pods constantly crash. I took a deep look into the ...
- 143
4
votes
1
answer
1k
views
How can we modify the heartbeat synchronization time of the etcd cluster on Kubernetes
I want to modify heartbeat-interval of embedded etcd cluster running on kubernetes, I know while installation we can do with kubeadm extraargs but how to do in running cluster?
Before modifying I want ...
- 190
3
votes
2
answers
5k
views
etcd2 fails to start on my CoreOS node
I am trying to start etcd2 in my CoreOS node.
I have this in my cloud-config:
coreos:
etcd2:
discovery: https://discovery.etcd.io/new?size=1
advertise-client-urls: http://127.0.0.1:2379,...
- 441
3
votes
2
answers
821
views
What exactly is Kubernetes API datastore?
THis page mentioned Kubernetes API datastore -- https://projectcalico.docs.tigera.io/getting-started/kubernetes/flannel/flannel -- and it seems to suggest that Kubernetes API datastore is a ...
- 141
3
votes
1
answer
5k
views
missing '=' etcd when defining service file
I'm struggling while following Kelsey Hightower's "Kubernetes the Hard Way" tutorial. I've gone off script, because I'm trying to bootstrap k8s on a local server.
I've got the point where I'm ...
- 61
3
votes
1
answer
2k
views
PostgreSQL failover with Kubernetes
I've got master slave replication working with Kubernetes, but would now like to implement failover. I have pods running with the service=postgresql and role=master or role=slave roles. When the ...
3
votes
0
answers
1k
views
k3s without HA: how to switch master node?
Rancher documentation on k3s is quite nice and its HA support (both with external DB or embedded etcd) look nice, but I don't want/need an HA setup.
In case my master node fails, I don't mind having ...
- 123
3
votes
0
answers
499
views
Weighting an etcd instance to have two quorum votes
For obvious reasons, best practice in building clusters that require distributed consensus is to use 3, 5, or another node count that has a tiebreaker in place.
Sometimes, however, one has two ...
- 986
3
votes
0
answers
288
views
etcd cluster setup with chef - determining initial-cluster-state
I am writing chef scripts to setup an etcd cluster. The list of nodes (including their IPs) will be hardcoded in the chef script (as an attribute) and it will pass them to etcd using static ...
- 201
2
votes
1
answer
457
views
Why do kubernetes kube-api server need etcd-keyfile and kubelet-client-key
As I understand kube-api server acts as a client when communicating with ETCD and Kubelet. Both ETCD and Kubelet act as servers for kube-api. With secure environment (two way SSL authentication), kube-...
- 123
2
votes
1
answer
2k
views
etcd2 fails on my CoreOS with my very simple setup
I want to setup a 3 nodes cluster running my own discovery service and I am following this simple guide for static discovery. I know my 3 machines IP addresses and here is when I try to start etcd2 on ...
- 441
2
votes
1
answer
265
views
Resizing Digital Ocean Droplets that use CoreOS break the CoreOs Cluster
I'm new to CoreOs and I have been experimenting with it on DigitalOcean. Let me start out by saying, I'm not sure if this is a DigitalOcean issue or CoreOS issue.
How to replicate:
Spin up 2 CoreOS ...
- 151
2
votes
2
answers
5k
views
Kubernetes - kubeadm join - Connection refused after new control plane joined
Issue
I am trying to join a second control-plane node to a K8S cluster. The first node has a IP of 10.1.50.4 where the second node has a IP of 10.1.50.5. There is a load balancer Virtual IP for the ...
- 320
2
votes
1
answer
99
views
Why doesn't CoreOS start etcd2 automatically, even though my cloud-config says so?
I am running CoreOS beta 991.1.0 on a Google Compute Engine instance, where cloud-config is configured via the user_data metadata variable. My cloud-config looks like this:
#cloud-config
write_files:
...
google-compute-engine
- 2,145
2
votes
0
answers
780
views
Changing out all Control Plane Nodes safely - Best Practice
I have a 3 control plane node HA kubeadm cluster that I need to completely replace. I have already replaced the worker nodes. How do I completely replace the control plane without downtime?
I use a LB ...
- 21
2
votes
0
answers
334
views
Kubernetes on Flatcar Linux will not start up kube API server after a reboot of the master node
Thank you for reading and taking your time to review this problem.
I have a problem using my Kubernetes cluster.
It is running Flatcar Linux, made by Kinvolk, recently acquired by Microsoft. I have ...
- 91
2
votes
0
answers
806
views
Kubernetes V1.19.13 - kubeapi servers not able connecting to different etcd database
I have upgraded Kubernets cluster ( 3 master, 3 etcd servers database) from 1.18 to v1.19.13 and ETCD to 3.4.13. since than API servers are not stable, keep switching different etcd server, because of ...
- 243
2
votes
1
answer
497
views
CoreOS 1353.4.0 - How should I configure Ignition for an etcd cluster member with dynamic discovery?
Using CoreOS 1353.4.0, on DigitalOcean, how should I write an Ignition config for a machine that is to join an etcd cluster with dynamic discovery?
I've written the following Ignition config (...
- 2,145
2
votes
1
answer
1k
views
problems configuring etcd2 for TLS
I have at home two CoreOS stable v1122.2.0 machines. both of them are registered at discovery.etcd.io properly. (i see both of them registered at https://ip:2380 when i browse to the discovery url).
...
- 333
2
votes
1
answer
561
views
Docker container can't communicate with etcd
I created 3 coreos machines at digitalocean using this cloud-config.yml: https://gist.github.com/socketwiz/d6fe23d19180a1ad8b5a where token was replaced with the one I retrieved from:
$ curl -w "\n" ...
- 121
1
vote
1
answer
3k
views
Removing a stale CoreOS node from discovery.etcd.io
I removed a node from a CoreOS cluster and brought up a fresh one in its place. Unfortunately, it picked up the same IP as the old machine.
This stopped etcd from discovering the rest of the cluster, ...
- 1,576
1
vote
2
answers
5k
views
How to recover from master failure in kubernetes
I have three nodes multi-master kubernetes(1.17.3) cluster(Stacked control plane and etcd nodes),
11.11.11.1 - master1
11.11.11.2 - master2
11.11.11.3 - master3
So before going to productions, I am ...
- 190
1
vote
2
answers
564
views
Own etcd cluster for Kubernetes
I want to build my own Kubernetes cluster across two locations (300 km distance) and integrate it into GitLab.
Let me list my ideas. My question is if I have a mistake in my thinking somewhere and ...
- 564
1
vote
3
answers
4k
views
kubernetes: how to check pod IP range allocated to node
Every node in a kubernetes cluster has a dedicated range of IP addresses that it can allocate to the pods.
How can one check what that IP range is?
1
vote
2
answers
4k
views
etcdctl endpoint health --> unhealty?
If I run etcdctl it hangs. If I use --debug=true I see:
root@k8scp:~# kubectl exec -n kube-system -it etcd-k8scp sh
sh-5.1# ETCDCTL_API=3 etcdctl --debug=true endpoint health
{"level":&...
- 3,893
1
vote
2
answers
1k
views
"EtcdException: Could not get the list of servers" while running calico rkt container on coreos
I have two coreos stable v1122.2.0 machines, each one with etcd2 configured with tls.
I created the certificates using https://github.com/coreos/etcd/tree/master/hack/tls-setup.
now I'm trying to ...
- 333
1
vote
1
answer
4k
views
kubelet not able to update node status
kubernetes node status reports as below.
root@csp-nso-server78:~# kubectl get nodes 10.213.5.78
NAME STATUS AGE
10.213.5.78 NotReady 1h
The kubelet logs show below error
E0812 11:...
- 151
1
vote
2
answers
502
views
ETCD restore granularity
I'm studying some backup and restore solutions for Kubernetes and Openshift, typically with Velero. So, I'm not familiar with ETCD backups, but I would like to ask about the granularity level of etcd ...
- 11
1
vote
1
answer
2k
views
How to start etcd in docker from systemd?
I want to start etcd (single node) in docker from systemd, but something seems to go wrong - it gets terminated about 30 seconds after start.
It looks like the service starts in status "...
- 1,207
1
vote
1
answer
1k
views
ETCD Cluster configuration for Kubernetes: Which one should be considered?
I would like to know how to deploy a ETCD cluster for Kubernetes.
It seems like there are two different documentation and I don't know which one must be considered or the impact of each of them.
From ...
- 123
1
vote
3
answers
13k
views
Kubernetes OCI runtime exec failed - starting container process caused "exec: \"etcdctl\": executable file not found in $PATH": unknown
Background
Created a fresh Kubernetes cluster using kubeadm init --config /home/kube/kubeadmn-config.yaml --upload-certs and then joining the 2nd control plane node by running the below.
kubeadm join ...
- 71
1
vote
1
answer
259
views
Multiple Core-OS clusters with a single etcd cluster?
Is it possible to create different core os clusters all linked up to 1 etcd cluster ? If yes, how ?
I have a scenario where I need to create multiple clusters but I want to keep one etcd cluster. ...
- 113
1
vote
1
answer
52
views
Issuing a single command from a cluster of servers?
I have a cluster of servers that will be running a simple ruok command with fallover. The command will be something simple like curl -X POST http://foo.com/status. Due to the fact that I will have ...
- 607
1
vote
1
answer
1k
views
ETCD fails to start, running CoreOS on AWS,EC2
I use ami-05ffb06f, t2.micro EC2 instance.
I use cloud-config to start etcd, but it fails to start
Error: client: etcd cluster is unavailable or misconfigured
error #0: dial tcp 127.0.0.1:4001: ...
1
vote
3
answers
2k
views
Kubernetes, flannel and exposing services
I have a kubernetes setup running nicely, but I can't seem to expose services externally. I'm thinking my networking is not set up correctly:
kubernetes services addresses: --service-cluster-ip-range=...
- 626
1
vote
3
answers
701
views
Recover from failed etcd2 / CoreOS cluster
I've got a cluster of 3 CoreOS machines running on Azure. I rebooted two at the same time and the cluster failed as expected.
I need to replace the discovery token, cloud-config is read on every ...
- 33
1
vote
0
answers
5k
views
Error from server: etcdserver: request timed out - error after etcd backup and restore
I have done the etcd backup and then a restore on the same cluster and now I'm having these issues where I can list resources but I can't create or delete.
It's a 1 master and 2 workers setup , ...
- 121
1
vote
0
answers
1k
views
ETCD snapshots causing etcdserver: leader changed
For a while now we've experienced regular errors from operations on kube API in AKS resulting in etcdserver: leader changed message. From what we've learned there is an ETCD snapshot performed every ...
1
vote
1
answer
689
views
systemd terminates etcd service started with podman - reception only permitted for main PID
I try to start etcd as a systemd service run in a container with podman.
After start I get this error log from systemd:
systemd[1]: etcd.service: Got notification message from PID 4696, but reception ...
- 1,207
1
vote
0
answers
651
views
How to spin up a ETCD docker container and restore a backup obtained from a kubernetes ETCD Cluster node?
I am trying to spin up a ETCD node in docker and restore a backup obtained from the Kubernetes Cluster node.
ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --user=root:'root' snapshot save ...
1
vote
1
answer
4k
views
TLS handshake issues with etcd
We are using external etcd cluster for k8s cluster. We connected master to this etcd server but receive
"tls: first record does not look like a TLS handshake"
How to fix this issues? (for ...
1
vote
0
answers
939
views
Etcd: machines can't connect to cluster
I reconfigurated my etcd cluster for working with certificates and now when i trying to start i'm getting
First machine:
Nov 05 16:24:43 Dm3Ch-Server-Node1 etcd[13999]: 21b9ef35413b514 received vote ...
- 111
1
vote
1
answer
496
views
SSH fails on CoreOS when substituting etcd cloud-config variables
I have a minimal cloud-config that works without problems on DigitalOcean. I added some hardening for SSH, which requires restarting sshd.socket to become effective:
units:
- name: sshd.socket
...
- 111
1
vote
2
answers
296
views
Does a CoreOS member need to have a public ip to join an etcd2 cluster?
I am starting 5 coreos EC2 members all in a private subnet.
Then assign one elastic ip to one of the members.
It seems that only that one with an ip assigned can join the etcd2 cluster, and is ...
- 111
1
vote
1
answer
276
views
Kubernetes service etcd références
If someone with quite good kubernetes's services understanding could confirm me that once you require a new service to be created kubernetes push a reference of it on the etcd database and what is the ...
- 975
1
vote
0
answers
110
views
Configure cloud-config using Azure Powershell 1.0
I have spent some significant time getting an Azure PS script starting up a few CoreOS instances. I have everything working except I have no clue how to get the cloud-config added.
Using the CLI, ...
- 123
0
votes
2
answers
1k
views
CoreOS: Inject a real private IP in etcd2 cloud-config
I have three networks on Rackspace:
Public network
Service network 10.181.XXX.XXX (they call it a private network sometimes but it is not private per-se, it is datacenter-wide private so their ...
- 129
0
votes
1
answer
211
views
etcdctl: unknown command "save" for "etcdctl"
I entered the etcd container:
kubectl -n kube-system exec -it etcd-k8scp -- sh
The I try to backup the container like explained in the K8s docs
ETCDCTL_API=3 etcdctl --endpoints $ENDPOINT snapshot ...
- 3,893
Related Tags
kubernetes × 48
coreos × 20
linux × 5
docker × 5
ubuntu × 3
azure × 3
cluster × 3
systemd × 3
kubeadm × 3
kubectl × 3
debian × 2
cloud-init × 2
zookeeper × 2
containerd × 2
patroni × 2
networking × 1
ssh × 1
ssl × 1
iptables × 1
vpn × 1
powershell × 1
backup × 1
more related tags
Hot Network Questions
- Different Bekenstein bound equations – what’s the difference?
- Can I tell a MILP solver to prefer solutions with fewer fractions?
- Tombs of Ancients
- Could a transparent frequency-altering material be possible?
- Visit USA via land border for French citizen
- Simple Container Class
- How to make D&D easier for kids?
- White grids appears when export GraphicsRow to PDF
- How can I confirm for sure that a CVE has been mitigated on a RHEL system?
- Reconstructing Euro results
- QGIS Labeling expression to format a values list in a series of 2 columns of 2 records
- Cleaning chain a few links at a time
- DSP Puzzle: Advanced Signal Forensics
- Weird behavior by car insurance - is this legit?
- Next date in the future such that all 8 digits of MM/DD/YYYY are all different and the product of MM, DD and YY is equal to YYYY
- Were there engineers in blimp nacelles, and why were they there?
- Why Owasp-crs does not allow Content-Type: application/x-www-form-urlencoded
- Cancellation of the Deutschlandticket
- Trying to determine what this small glass-enclosed item is
- What could explain that small planes near an airport are perceived as harassing homeowners?
- What kind of sequence is between an arithmetic and a geometric sequence?
- Are both vocal cord and vocal chord correct?
- How do I get my D&D group to engage to a minimum
- What's Wrong With My Math - Odds of 3 Cards of the Same Suit When Drawing 10 Cards