Introduction
This script automates the process of updating Docker image tags in Helm values files by fetching the latest tags from DockerHub for multiple repositories. It interacts with DockerHub’s API, retrieves the latest version tags, and updates the corresponding YAML files located in a Helm chart.
The script ensures that Kubernetes deployments always use the latest image versions, which is crucial for maintaining up-to-date microservices in a Kubernetes environment.
Prerequisites
Before you can use this script, make sure you have
python3
pip install requests pyyaml
Docker and DockerHub Access Token
Create Docker-Secret:- Creating a Docker secret is necessary when you want to pull private Docker images from a Docker registry (like DockerHub or other container registries) in your Kubernetes environment. This is especially relevant if you are deploying services through Helm or directly with Kubernetes manifests and need access to private repositories.
kubectl --kubeconfig=./file.yaml(kubeconfig file name) create secret docker-registry docker-secret \ --docker-server=docker.io \ --docker-username=your-username \ --docker-password=your-password \ --docker-email=your-email \ --namespace=your-namespace nameHelm
Kubernetes Cluster (kubectl)
What is Helm?
Helm is an open-source package manager for Kubernetes, the popular container orchestration platform. Similar to yum but for Kubernetes. It bundles all related manifests(such as deployment, service, etc) into a chart. Helm helps you manage Kubernetes applications by providing a way to define, install, and upgrade complex Kubernetes applications.
In this context, we are using Helm to manage deployments of microservices, with each microservice’s configuration stored in a values.yaml file.
Why Use Helm in Kubernetes?
Helm simplifies Kubernetes deployments by:
Packaging all Kubernetes resources like Pods, Services, ConfigMaps, etc., into a single chart.
Allowing version control and rollback of Kubernetes applications.
Using values files to customize deployments easily.
Simplifying updates to applications by managing Kubernetes objects under a single release.
Helm allows us to maintain a standard deployment process, making it easier to modify, install, or update our applications in a Kubernetes environment.
Overview of the Script
This script performs the following tasks:
Fetch All Repositories from DockerHub: It uses the DockerHub API to fetch all repositories under a specific account.
Fetch the Highest
dev-*Tag for Each Repository: For each repository, the script fetches the list of available tags and identifies the latestdev-*tag.Update Helm Values File: The script looks for a Helm
values.yamlfile associated with the Docker repository and updates the image tag to the latestdev-*tag.Success Confirmation: It prints out success messages for updated files.
Creating a Namespace for Helm and Helm Chart
A namespace in Kubernetes is a way to logically separate resources. You should create a dedicated namespace for your Helm release to isolate the application.
kubectl create namespace development
helm create project-01
Let’s cd into the generated chart directory.
We’ll edit the files one by one according to our deployment requirements.
There are multiple files in templates directory created by Helm. We can customize our helm-chart according to our projects.
Let’s remove all default files from the template directory.
rm -rf templates/*
Create a deployment.yaml file and copy the following contents.
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Values.deployment.name }}
namespace: {{ .Values.namespace }}
spec:
replicas: {{ .Values.deployment.replicas }}
selector:
matchLabels:
app: {{ .Values.deployment.appLabel }}
template:
metadata:
labels:
app: {{ .Values.deployment.appLabel }}
spec:
containers:
- name: {{ .Values.deployment.containerName }}
image: {{ .Values.image.repository }}:{{ .Values.image.tag }} # This will pull the correct image and tag
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- containerPort: {{ .Values.deployment.containerPort }}
imagePullSecrets:
- name: {{ .Values.imagePullSecret }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ .Values.service.name }}
namespace: {{ .Values.namespace }}
spec:
selector:
app: {{ .Values.deployment.appLabel }}
ports:
- protocol: TCP
port: {{ .Values.service.port }}
targetPort: {{ .Values.service.targetPort }}
type: ClusterIP
values
The values.yaml file contains all the values that need to be substituted in the template directives we used in the templates. For example, deployment.yaml template contains a template directive to get the image repository, tag, and pullPolicy from the values.yaml file.
Now, create a values directory and store all the values on the basis of your project.
deployment:
appLabel: ******
containerName: *********
containerPort: **
name: ********
replicas: ***
image:
pullPolicy: Always
repository: *********
tag: ****
imagePullSecret: ******
namespace: ******
service:
name: ********
port: **
targetPort: **
Now we have the project-01 helm chart ready.
Validate the Helm Chart
Now to make sure that our chart is valid and, all the indentations are fine, we can run the below command. Ensure you are inside the chart directory.
helm lint .
If there is no error or issue, it will show this result
==> Linting ./project-01
[INFO] Chart.yaml: icon is recommended
1 chart(s) linted, 0 chart(s) failed
To validate if the values are getting substituted in the templates, you can render the templated YAML files with the values using the following command. It will generate and display all the manifest files with the substituted values.
helm template .
Deploy the Helm Chart
When you deploy the chart, Helm will read the chart and configuration values from the values.yaml file and generate the manifest files. Then it will send these files to the Kubernetes API server, and Kubernetes will create the requested resources in the cluster.
Execute the following command
helm install my-release ./project-01 -f ./values -n development
Verify Deployment
kubectl get all -n development
Since we have many values files, manually updating each one every time the Docker image tag changes is very time-consuming. Therefore, we have a Python script that automatically updates the image tags in the values files whenever the Docker image is updated.
import requests
import yaml
import os
import sys
# DockerHub credentials
username = "write-your-dockerhub-username"
token = input("Enter your DockerHub Access Token: ")
# Path to your values directory
values_directory = "./values"
# Function to fetch the highest 'dev-*' tag for a repository
def fetch_highest_dev_tag(repo_name):
url = f"https://hub.docker.com/v2/repositories/{username}/{repo_name}/tags"
headers = {
'Authorization': f'Bearer {token}'
}
response = requests.get(url, headers=headers, params={"page_size": 100})
if response.status_code == 200:
data = response.json()
dev_tags = [tag['name'] for tag in data['results'] if tag['name'].startswith('dev-')]
if dev_tags:
highest_tag = max(dev_tags, key=lambda x: int(x.split('-')[1]))
return highest_tag
else:
return None
else:
print(f"Failed to fetch tags for {repo_name}")
return None
# Function to update the image tag in the values file
def update_values_file(repo_name, new_tag):
# Construct the expected values filename based on the repository name
values_file = os.path.join(values_directory, f"{repo_name}.yaml") # Adjusted here
if os.path.exists(values_file):
with open(values_file, 'r') as file:
values_data = yaml.safe_load(file)
if 'image' in values_data and 'tag' in values_data['image']:
old_tag = values_data['image']['tag']
values_data['image']['tag'] = new_tag
with open(values_file, 'w') as file:
yaml.safe_dump(values_data, file)
print(f"Successfully updated {values_file} from {old_tag} to {new_tag}.")
else:
print(f"No image.tag found in {values_file}")
else:
print(f"Values file for {repo_name} not found.")
# Function to fetch all repositories under the DockerHub account
def fetch_all_repositories():
repositories = []
page = 1
while True:
url = f"https://hub.docker.com/v2/repositories/{username}/"
headers = {
'Authorization': f'Bearer {token}'
}
response = requests.get(url, headers=headers, params={"page": page, "page_size": 100})
if response.status_code == 200:
data = response.json()
repositories.extend([repo['name'] for repo in data['results']])
if not data['next']:
break
page += 1
else:
break
return repositories
# Main logic to update tags
def main():
if len(sys.argv) == 1:
# No arguments passed, update all repositories
print("Updating all repositories...")
repositories = fetch_all_repositories()
for repo_name in repositories:
highest_tag = fetch_highest_dev_tag(repo_name)
if highest_tag:
update_values_file(repo_name, highest_tag)
elif len(sys.argv) == 2:
# Single repository mode
repo_name = sys.argv[1]
print(f"Updating single repository: {repo_name}")
highest_tag = fetch_highest_dev_tag(repo_name)
if highest_tag:
update_values_file(repo_name, highest_tag)
else:
print(f"No 'dev-*' tags found for {repo_name}")
else:
print("Usage: python3 script.py [repository_name]")
sys.exit(1)
if __name__ == "__main__":
main()
To run the script we should run following command
python3 script.py // for all repository
python3 script.py microservice-name // for single repository