Configure service accounts for pods

User accounts versus service accounts

Kubernetes distinguishes between the concept of a user account and a service account
for a number of reasons:

• User accounts are for humans. Service accounts are for processes, which run
  in pods.
• User accounts are intended to be global. Names must be unique across all
  namespaces of a cluster. Service accounts are namespaced.
• Typically, a cluster’s user accounts might be synced from a corporate
  database, where new user account creation requires special privileges and is
  tied to complex business processes. Service account creation is intended to be
  more lightweight, allowing cluster users to create service accounts for
  specific tasks by following the principle of least privilege.
• Auditing considerations for humans and service accounts may differ.
• A config bundle for a complex system may include definition of various service
  accounts for components of that system. Because service accounts can be created
  without many constraints and have namespaced names, such config is portable.

Managed PKI for Effective Certificate Management

For many organizations, working with a 3rd party that manages a PKI for you is the easiest option, considering the complexities of PKI management. SecureW2 provides all the tools needed to launch a turnkey PKI for easy certificate configuration, distribution, and management. And of course, our certificates are trusted for external use and eliminate the x509 certificate error.

SecureW2’s cloud-based PKI integrates easily with any network infrastructure and is completely vendor-neutral. And because it’s based in the cloud, it offers the innumerable benefits of cloud technology: easy configuration, no physical installation, lower management costs over time, future-proofed, etc.

Our comprehensive management tools allow for a huge amount of flexibility for admins. It provides a centralized place to manage the entire certificate lifecycle and never fall behind certificate expiration dates. Additionally, it allows for insight into authentication events and remote troubleshooting should any network-related issues arise.

Certificates distributed from SecureW2’s managed PKI can be used for SSL, S/MIME, RADIUS authentication, VPN, web app authentication, and more. If this is your first foray into using certificates and you determine they should be used elsewhere on the network, SecureW2 can create an easy transition into greater certificate usage.

Privilege escalation via workload creation or edits

Users who can create/edit pods in a namespace, either directly or through a controller
such as an operator, could escalate their privileges in that namespace.

Caution: System administrators, use care when granting access to create or edit workloads.
Details of how these can be misused are documented in

Escalation paths

• Mounting arbitrary secrets in that namespace
  • Can be used to access secrets meant for other workloads
  • Can be used to obtain a more privileged service account’s service account token
• Using arbitrary Service Accounts in that namespace
  • Can perform Kubernetes API actions as another workload (impersonation)
  • Can perform any privileged actions that Service Account has
• Mounting configmaps meant for other workloads in that namespace
• Mounting volumes meant for other workloads in that namespace

Caution: System administrators should be cautious when deploying CRDs that
change the above areas. These may open privilege escalations paths.
This should be considered when deciding on your RBAC controls.

Kubelet client certificate rotation fails

By default, kubeadm configures a kubelet with automatic rotation of client certificates by using the symlink specified in .
If this rotation process fails you might see errors such as
in kube-apiserver logs. To fix the issue you must follow these steps:

1. Backup and delete and from the failed node.

2. From a working control plane node in the cluster that has execute
   .
   must be set to the name of the existing failed node in the cluster.
   Modify the resulted manually to adjust the cluster name and server endpoint,
   or pass (it accepts ). If your cluster does not have
   the you must sign the embedded certificates in the externally.

3. Copy this resulted to on the failed node.

4. Restart the kubelet () on the failed node and wait for
   to be recreated.

5. Manually edit the to point to the rotated kubelet client certificates, by replacing
   and with:

6. Restart the kubelet.

7. Make sure the node becomes .

Check certificate expiration

You can use the subcommand to check when certificates expire:

The output is similar to this:

The command shows expiration/residual time for the client certificates in the folder and for the client certificate embedded in the KUBECONFIG files used by kubeadm (, and ).

Additionally, kubeadm informs the user if the certificate is externally managed; in this case, the user should take care of managing certificate renewal manually/using other tools.

Warning: cannot manage certificates signed by an external CA.

Note: is not included in the list above because kubeadm configures kubelet
for automatic certificate renewal
with rotatable certificates under .
To repair an expired kubelet client certificate see
.

Warning:

On nodes created with , prior to kubeadm version 1.17, there is a
bug where you manually have to modify the contents of . After finishes, you should update to point to the
rotated kubelet client certificates, by replacing and with:

Service account automation

Three separate components cooperate to implement the automation around service accounts:

• A admission controller
• A Token controller
• A controller

ServiceAccount Admission Controller

The modification of pods is implemented via a plugin
called an Admission Controller.
It is part of the API server.
It acts synchronously to modify pods as they are created or updated. When this plugin is active
(and it is by default on most distributions), then it does the following when a pod is created or modified:

1. If the pod does not have a set, it sets the to .
2. It ensures that the referenced by the pod exists, and otherwise rejects it.
3. It adds a to the pod which contains a token for API access if neither the ServiceAccount nor the Pod’s is set to .
4. It adds a to each container of the pod mounted at , if the previous step has created a volume for ServiceAccount token.
5. If the pod does not contain any , then of the are added to the pod.

Bound Service Account Token Volume

FEATURE STATE:

The ServiceAccount admission controller will add the following projected volume instead of a Secret-based volume for the non-expiring service account token created by Token Controller.

This projected volume consists of three sources:

1. A ServiceAccountToken acquired from kube-apiserver via TokenRequest API. It will expire after 1 hour by default or when the pod is deleted. It is bound to the pod and has kube-apiserver as the audience.
2. A ConfigMap containing a CA bundle used for verifying connections to the kube-apiserver. This feature depends on the feature gate, which publishes a «kube-root-ca.crt» ConfigMap to every namespace. feature gate is graduated to GA in 1.21 and default to true. (This flag will be removed from —feature-gate arg in 1.22)
3. A DownwardAPI that references the namespace of the pod.

See more details about projected volumes.

Token Controller

TokenController runs as part of . It acts asynchronously. It:

• watches ServiceAccount creation and creates a corresponding
  ServiceAccount token Secret to allow API access.
• watches ServiceAccount deletion and deletes all corresponding ServiceAccount
  token Secrets.
• watches ServiceAccount token Secret addition, and ensures the referenced
  ServiceAccount exists, and adds a token to the Secret if needed.
• watches Secret deletion and removes a reference from the corresponding
  ServiceAccount if needed.

You must pass a service account private key file to the token controller in
the using the
flag. The private key is used to sign generated service account tokens.
Similarly, you must pass the corresponding public key to the
using the flag. The public key will be used to
verify the tokens during authentication.

To create additional API tokens

A controller loop ensures a Secret with an API token exists for each
ServiceAccount. To create additional API tokens for a ServiceAccount, create a
Secret of type with an annotation
referencing the ServiceAccount, and the controller will update it with a
generated token:

Below is a sample configuration for such a Secret:

Not possible to pass a comma separated list of values to arguments inside a —component-extra-args flag

flags such as allow you to pass custom arguments to a control-plane
component like the kube-apiserver. However, this mechanism is limited due to the underlying type used for parsing
the values ().

If you decide to pass an argument that supports multiple, comma-separated values such as
this flag will fail with
. This happens because the list of arguments for
expects pairs and in this case is considered
as a key that is missing a value.

Alternatively, you can try separating the pairs like so:

but this will result in the key only having the value of .

A known workaround is to use the kubeadm configuration file.

Add ImagePullSecrets to a service account

Add image pull secret to service account

Next, modify the default service account for the namespace to use this secret as an imagePullSecret.

You can instead use , or manually edit the YAML manifests as shown below:

The output of the file is similar to this:

Using your editor of choice (for example ), open the file, delete line with key , add lines with and save.

The output of the file is similar to this:

Finally replace the serviceaccount with the new updated file

Verify imagePullSecrets was added to pod spec

Now, when a new Pod is created in the current namespace and using the default ServiceAccount, the new Pod has its field set automatically:

The output is:

Non-public IP used for containers

In some situations and commands may return with the following errors in an otherwise functional cluster:

• This may be due to Kubernetes using an IP that can not communicate with other IPs on the seemingly same subnet, possibly by policy of the machine provider.

• DigitalOcean assigns a public IP to as well as a private one to be used internally as anchor for their floating IP feature, yet will pick the latter as the node’s instead of the public one.

  Use to check for this scenario instead of because will not display the offending alias IP address. Alternatively an API endpoint specific to DigitalOcean allows to query for the anchor IP from the droplet:

  The workaround is to tell which IP to use using .
  When using DigitalOcean, it can be the public one (assigned to ) or
  the private one (assigned to ) should you want to use the optional
  private network. The section of the kubeadm

  

  

  

  

  

  

  

  

  

  

  can be used for this.

  Then restart :

Harbor docker login x509 certificate signed by unknown authority

s http-equiv=»Content-Type» content=»text/html;charset=UTF-8″>tyle=»clear:both;»>

Предисловие

После настройки частного реестра Docker с помощью Harbour на CentOS7 и настройки доступа HTTPS используйтеПроизошла ошибка:

Чтобы решить эту проблему, мы должны позволить Docker доверять сгенерированному нами сертификату ЦС, аналогично импорту и доверию корневому сертификату ЦС в браузере.

Сгенерированный сертификат

имя файла	Описание
ca.crt	Открытый ключ сертификата ЦС
ca.key	Ключ сертификата ЦС
harbor.crt	Открытый ключ SSL-сертификата Harbor
harbor.key	Ключ сертификата Harbor SSL

Справочные документы

• https://github.com/goharbor/harbor/blob/master/docs/configure_https.md
• https://manuals.gfi.com/en/kerio/connect/content/server-configuration/ssl-certificates/adding-trusted-root-certificates-to-the-server-1605.html
• https://github.com/moby/moby/issues/8849
• https://blog.csdn.net/hjmhz/article/details/81912315

Интеллектуальная рекомендация

19.03.21 Я загрузил комплексные обучающие видеоуроки Photoshop CC 2015 и обучающие видеоуроки по новым функциям PS CC 2015. Я просмотрел несколько видео, но мне кажется, что они в основном объясняют н…

…

проверка данных весеннего mvc Два способа проверки данных Spring MVC: 1.JSR303 2.Hibernate Validator Второй метод является дополнением к первому методу Шаги для проверки данных с использованием Hibern…

Существует два способа вызова между сервисами Springcloud: RestTemplate и Feign. Здесь мы представляем сервисы вызова RestTemplate. 1. Что такое RestTemplate RestTemplate — это структура веб-запросов …

1. Понимать предварительный, средний, последующий порядок и иерархическую последовательность бинарных деревьев; Свяжите язык C со структурой данных двоичного дерева; Освойте с…

Вам также может понравиться

Последнее обучение, как использовать Kaldi, чтобы проснуться без использования WSTF, поэтому вам нужно глубоко пойти в Kaldi для обучения. Временное состояние обучения. Три изображения представляют со…

Во время простоя некоторые веб-страницы, которые мы создали, не были завершены, но не хотят, чтобы другие видели, вы можете создать простой эффект шифрования страницы на странице этой веб-страницы, ан…

Расширенные статьи серии Zookeeper 1. NIO, ZAB соглашение, 2PC представления концепции 2. Лидер выборов 3. Рукописный распределенный замок, центр настройки ==================================== 1. NIO,…

Посмотрите на конечный эффект первым DemoPreview.gif SETP1 эффект капли воды Первая реакция на эффект капли воды — нарисовать замкнутую кривую. С помощью события MotionEvent измените радиус во время п…

…

Review Your Request Attributes

Kubernetes reviews only the following API request attributes:

• user — The string provided during authentication.
• group — The list of group names to which the authenticated user belongs.
• extra — A map of arbitrary string keys to string values, provided by the authentication layer.
• API — Indicates whether the request is for an API resource.
• Request path — Path to miscellaneous non-resource endpoints like or .
• API request verb — API verbs like , , , , , , , and are used for resource requests. To determine the request verb for a resource API endpoint, see .
• HTTP request verb — Lowercased HTTP methods like , , , and are used for non-resource requests.
• Resource — The ID or name of the resource that is being accessed (for resource requests only) — For resource requests using , , , and verbs, you must provide the resource name.
• Subresource — The subresource that is being accessed (for resource requests only).
• Namespace — The namespace of the object that is being accessed (for namespaced resource requests only).
• API group — The being accessed (for resource requests only). An empty string designates the core .

kube-proxy scheduled before node is initialized by cloud-controller-manager

In cloud provider scenarios, kube-proxy can end up being scheduled on new worker nodes before
the cloud-controller-manager has initialized the node addresses. This causes kube-proxy to fail
to pick up the node’s IP address properly and has knock-on effects to the proxy function managing
load balancers.

The following error can be seen in kube-proxy Pods:

A known solution is to patch the kube-proxy DaemonSet to allow scheduling it on control-plane
nodes regardless of their conditions, keeping it off of other nodes until their initial guarding
conditions abate:

The tracking issue for this problem is here.

Normal user

A few steps are required in order to get a normal user to be able to
authenticate and invoke an API. First, this user must have a certificate issued
by the Kubernetes cluster, and then present that certificate to the Kubernetes API.

Create private key

The following scripts show how to generate PKI private key and CSR. It is
important to set CN and O attribute of the CSR. CN is the name of the user and
O is the group that this user will belong to. You can refer to
RBAC for standard groups.

Create CertificateSigningRequest

Create a CertificateSigningRequest and submit it to a Kubernetes Cluster via kubectl. Below is a script to generate the CertificateSigningRequest.

Some points to note:

• has to be »
• could be made longer (i.e. for ten days) or shorter (i.e. for one hour)
• is the base64 encoded value of the CSR file content.
  You can get the content using this command:

Use kubectl to create a CSR and approve it.

Get the list of CSRs:

Approve the CSR:

Get the certificate

Retrieve the certificate from the CSR:

The certificate value is in Base64-encoded format under .

Export the issued certificate from the CertificateSigningRequest.

Create Role and RoleBinding

With the certificate created it is time to define the Role and RoleBinding for
this user to access Kubernetes cluster resources.

This is a sample command to create a Role for this new user:

This is a sample command to create a RoleBinding for this new user:

Add to kubeconfig

The last step is to add this user into the kubeconfig file.

First, you need to add new credentials:

Then, you need to add the context:

To test it, change the context to :

Pods in RunContainerError, CrashLoopBackOff or Error state

Right after there should not be any pods in these states.

• If there are pods in one of these states right after , please open an
  issue in the kubeadm repo. (or ) should be in the state
  until you have deployed the network add-on.
• If you see Pods in the , or state
  after deploying the network add-on and nothing happens to (or ),
  it’s very likely that the Pod Network add-on that you installed is somehow broken.
  You might have to grant it more RBAC privileges or use a newer version. Please file
  an issue in the Pod Network providers’ issue tracker and get the issue triaged there.
• If you install a version of Docker older than 1.12.1, remove the option
  when booting with and restart . You can see the MountFlags in .
  MountFlags can interfere with volumes mounted by Kubernetes, and put the Pods in state.
  The error happens when Kubernetes does not find files.

Signing

Control plane signer

The Kubernetes control plane implements each of the
,
as part of the kube-controller-manager.

Note: Prior to Kubernetes v1.18, the kube-controller-manager would sign any CSRs that
were marked as approved.

Note: The field was added in Kubernetes v1.22. Earlier versions of Kubernetes do not honor this field.
Kubernetes API servers prior to v1.22 will silently drop this field when the object is created.

API-based signers

Users of the REST API can sign CSRs by submitting an UPDATE request to the
subresource of the CSR to be signed.

As part of this request, the field should be set to contain the
signed certificate. This field contains one or more PEM-encoded certificates.

All PEM blocks must have the «CERTIFICATE» label, contain no headers,
and the encoded data must be a BER-encoded ASN.1 Certificate structure
as described in .

Example certificate content:

Non-PEM content may appear before or after the CERTIFICATE PEM blocks and is unvalidated,
to allow for explanatory text as described in section 5.2 of RFC7468.

When encoded in JSON or YAML, this field is base-64 encoded.
A CertificateSigningRequest containing the example certificate above would look like this:

Use Multiple Service Accounts.

Every namespace has a default service account resource called .
You can list this and any other serviceAccount resources in the namespace with this command:

The output is similar to this:

You can create additional ServiceAccount objects like this:

The name of a ServiceAccount object must be a valid
.

If you get a complete dump of the service account object, like this:

The output is similar to this:

then you will see that a token has automatically been created and is referenced by the service account.

You may use authorization plugins to .

To use a non-default service account, set the
field of a pod to the name of the service account you wish to use.

The service account has to exist at the time the pod is created, or it will be rejected.

You cannot update the service account of an already created pod.

You can clean up the service account from this example like this:

Using Flags for Your Authorization Module

You must include a flag in your policy to indicate which authorization module
your policies include:

The following flags can be used:

• Attribute-Based Access Control (ABAC) mode allows you to configure policies using local files.
• Role-based access control (RBAC) mode allows you to create and store policies using the Kubernetes API.
• WebHook is an HTTP callback mode that allows you to manage authorization using a remote REST endpoint.
• Node authorization is a special-purpose authorization mode that specifically authorizes API requests made by kubelets.
• This flag blocks all requests. Use this flag only for testing.
• This flag allows all requests. Use this flag only if you do not require authorization for your API requests.

You can choose more than one authorization module. Modules are checked in order
so an earlier module has higher priority to allow or deny a request.

Configure certificates manually

If you don’t want kubeadm to generate the required certificates, you can create them using a single root CA or by providing all certificates. See Certificates for details on creating your own certificate authority.
See Certificate Management with kubeadm for more on managing certificates.

Single root CA

You can create a single root CA, controlled by an administrator. This root CA can then create multiple intermediate CAs, and delegate all further creation to Kubernetes itself.

Required CAs:

path	Default CN	description
ca.crt,key	kubernetes-ca	Kubernetes general CA
etcd/ca.crt,key	etcd-ca	For all etcd-related functions
front-proxy-ca.crt,key	kubernetes-front-proxy-ca	For the front-end proxy

On top of the above CAs, it is also necessary to get a public/private key pair for service account management, and .
The following example illustrates the CA key and certificate files shown in the previous table:

All certificates

If you don’t wish to copy the CA private keys to your cluster, you can generate all certificates yourself.

Required certificates:

Default CN	Parent CA	O (in Subject)	kind	hosts (SAN)
kube-etcd	etcd-ca		server, client	, , ,
kube-etcd-peer	etcd-ca		server, client	, , ,
kube-etcd-healthcheck-client	etcd-ca		client
kube-apiserver-etcd-client	etcd-ca	system:masters	client
kube-apiserver	kubernetes-ca		server	, , ,
kube-apiserver-kubelet-client	kubernetes-ca	system:masters	client
front-proxy-client	kubernetes-front-proxy-ca		client

: any other IP or DNS name you contact your cluster on (as used by kubeadm
the load balancer stable IP and/or DNS name, , , ,
, )

where maps to one or more of the types:

kind	Key usage
server	digital signature, key encipherment, server auth
client	digital signature, key encipherment, client auth

Note: Hosts/SAN listed above are the recommended ones for getting a working cluster; if required by a specific setup, it is possible to add additional SANs on all the server certificates.

Note:

For kubeadm users only:

• The scenario where you are copying to your cluster CA certificates without private keys is referred as external CA in the kubeadm documentation.
• If you are comparing the above list with a kubeadm generated PKI, please be aware that , and certificates
  are not generated in case of external etcd.

Certificate paths

Certificates should be placed in a recommended path (as used by kubeadm).
Paths should be specified using the given argument regardless of location.

Default CN	recommended key path	recommended cert path	command	key argument	cert argument
etcd-ca	etcd/ca.key	etcd/ca.crt	kube-apiserver		—etcd-cafile
kube-apiserver-etcd-client	apiserver-etcd-client.key	apiserver-etcd-client.crt	kube-apiserver	—etcd-keyfile	—etcd-certfile
kubernetes-ca	ca.key	ca.crt	kube-apiserver		—client-ca-file
kubernetes-ca	ca.key	ca.crt	kube-controller-manager	—cluster-signing-key-file	—client-ca-file, —root-ca-file, —cluster-signing-cert-file
kube-apiserver	apiserver.key	apiserver.crt	kube-apiserver	—tls-private-key-file	—tls-cert-file
kube-apiserver-kubelet-client	apiserver-kubelet-client.key	apiserver-kubelet-client.crt	kube-apiserver	—kubelet-client-key	—kubelet-client-certificate
front-proxy-ca	front-proxy-ca.key	front-proxy-ca.crt	kube-apiserver		—requestheader-client-ca-file
front-proxy-ca	front-proxy-ca.key	front-proxy-ca.crt	kube-controller-manager		—requestheader-client-ca-file
front-proxy-client	front-proxy-client.key	front-proxy-client.crt	kube-apiserver	—proxy-client-key-file	—proxy-client-cert-file
etcd-ca	etcd/ca.key	etcd/ca.crt	etcd		—trusted-ca-file, —peer-trusted-ca-file
kube-etcd	etcd/server.key	etcd/server.crt	etcd	—key-file	—cert-file
kube-etcd-peer	etcd/peer.key	etcd/peer.crt	etcd	—peer-key-file	—peer-cert-file
etcd-ca		etcd/ca.crt	etcdctl		—cacert
kube-etcd-healthcheck-client	etcd/healthcheck-client.key	etcd/healthcheck-client.crt	etcdctl	—key	—cert

Same considerations apply for the service account key pair:

private key path	public key path	command	argument
sa.key		kube-controller-manager	—service-account-private-key-file
	sa.pub	kube-apiserver	—service-account-key-file

The following example illustrates the file paths you need to provide if you are generating all of your own keys and certificates: