Hi, I'm Bolo!
What would you like to know?
ASK BOLO
Please to access Bolo.

Table of Contents

Search

  1. Advanced clusters
  2. Setting up AWS
  3. Setting up Google Cloud
  4. Setting up Microsoft Azure
  5. Setting up a self-service cluster
  6. Setting up a local cluster
  7. Advanced configurations
  8. Troubleshooting
  9. Appendix A: Command reference

Advanced Clusters

Advanced Clusters

Create an optimized cluster role

Create an optimized cluster role

Create a cluster role that has permission to access and edit cluster resources. A cluster role is global and not associated with a namespace. If you need to restrict permissions to a specific namespace, you can split the cluster role permissions into two different roles.
The permissions required depend on whether the cluster runs mappings with the Spark shuffle service. The Spark shuffle service helps the cluster perform dynamic allocation for Spark jobs. The service is responsible for persisting shuffle files beyond the lifetime of the executors, allowing the number of executors to scale up and down without losing computation.

Minimum permissions to run a mapping with the Spark shuffle service

The following code snippet shows the minimum permissions required to run a mapping with the Spark shuffle service:
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: optimized-cluster-role rules: - apiGroups: [""] resources: ["services","pods","secrets","configmaps"] verbs: ["watch","list","get","create","update","patch","delete","deletecollection"] - apiGroups: ["apps"] resources: ["daemonsets","deployments"] verbs: ["watch","list","get","create","update","patch","delete","deletecollection"] - apiGroups: [""] resources: ["nodes"] verbs: ["watch","list","get","patch"] - apiGroups: [""] resources: ["namespaces","persistentvolumeclaims"] verbs: ["watch","list","get"]
The permissions in this code snippet applies to all namespaces.
If you need to limit the permissions to certain namespaces only, split these permissions into two roles: Role and ClusterRole. ClusterRole contains permissions for resources that are global, while Role contains permissions for resources that are specific to a namespace.
The following code snippet shows the permissions for Role:
apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: rbac-informatica-np-admin namespace: informatica rules: - apiGroups: [""] resources: ["services","pods","secrets","configmaps"] verbs: ["watch","list","get","create","update","patch","delete","deletecollection"] - apiGroups: ["apps"] resources: ["daemonsets","deployments"] verbs: ["watch","list","get","create","update","patch","delete","deletecollection"]
The following code snippet shows the permissions for ClusterRole:
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: rbac-informatica-global-admin rules: - apiGroups: [""] resources: ["nodes"] verbs: ["watch","list","get","patch"] - apiGroups: [""] resources: ["namespaces", "persistentvolumeclaims"] verbs: ["watch","list","get"]

Minimum permissions to run a mapping without the Spark shuffle service

The following code snippet shows the minimum permissions required to run a mapping without the Spark shuffle service:
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: optimized-cluster-role rules: - apiGroups: [""] resources: ["services","pods","secrets","configmaps"] verbs: ["watch","list","get","create","update","patch","delete","deletecollection"] - apiGroups: [""] resources: ["nodes"] verbs: ["watch","list","get"] - apiGroups: [""] resources: ["namespaces","persistentvolumeclaims"] verbs: ["watch","list","get"]
The permissions in this code snippet applies to all namespaces.
If you need to limit the permissions to certain namespaces only, split these permissions into two roles: Role and ClusterRole. ClusterRole contains permissions for resources that are global, while Role contains permissions for resources that are specific to a namespace.
The following code snippet shows the permissions for Role:
apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: rbac-informatica-np-admin namespace: informatica rules: - apiGroups: [""] resources: ["services","pods","secrets","configmaps"] verbs: ["watch","list","get","create","update","patch","delete","deletecollection"]
The following code snippet shows the permissions for ClusterRole:
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: rbac-informatica-global-admin rules: - apiGroups: [""] resources: ["nodes"] verbs: ["watch","list","get"] - apiGroups: [""] resources: ["namespaces", "persistentvolumeclaims"] verbs: ["watch","list","get"]

0 COMMENTS

We’d like to hear from you!