![]() The KubernetesPodOperator is entirely powered by the resources allocated to the Extra Capacity slider of your deployment's Configure page in the Software UI in lieu of needing a Celery worker (or scheduler resources for those running the Local Executor). This ensures that there are no unused pods in your cluster taking up resources.Īdd resources to your Deployment on Astronomer is_delete_operator_pod=True: This setting ensures that once a KubernetesPodOperator task is complete, the Kubernetes Pod that ran that task is terminated.This ensures that the Kubernetes Pod running your task has the correct permissions within the cluster. ![]() in_cluster=True: When this value is set, your task will run within the cluster from which it's instantiated.To pull an image from a private registry, read Pull images from a Private Registry. The value you specify is assumed to be an image tag that's publicly available on Docker Hub. image: This is the Docker image that the operator will use to run its defined task, commands, and arguments.Information about this namespace can be programmatically imported as long as you set this variable. namespace = conf.get("kubernetes", "NAMESPACE"): Every Deployment runs on its own Kubernetes namespace. ![]() Instantiate the operator based on your image and setup:įor each instantiation of the KubernetesPodOperator, you must specify the following values: Run the following command to import the KubernetesPodOperator:įrom .operators.kubernetes_pod import KubernetesPodOperator Pip install apache-airflow-providers-cncf-kubernetes Run the following command to install the apache-airflow-providers-cncf-kubernetes package: Set Up the KubernetesPodOperator Import the operator See Running KubernetesPodOperator locally. Note: If you haven't already, Astronomer recommends testing the KubernetesPodOperator in your local environment.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |