Error: INSTALLATION FAILED: timed out waiting for the condition
I’m setting it up on windows using WSL2.
I’ve reviewed every step of the process preceding this error, and verified that all outputs were as expected.
I’ll appreciate any help on the matter.
Upon running microk8s helm install cesium-ion ~/cesium-ion-1.0.0/cesium-ion/ --wait --values ~/cesium-ion-1.0.0/cesium-ion/values.yaml --namespace cesium-ion --create-namespace, we get the following error after about 5mins:
Error: INSTALLATION FAILED: timed out waiting for the condition
We’ve run sudo microk8s inspect and it returns no warnings.
Inspecting system
Inspecting Certificates
Inspecting services
Service snap.microk8s.daemon-cluster-agent is running
Service snap.microk8s.daemon-containerd is running
Service snap.microk8s.daemon-kubelite is running
Service snap.microk8s.daemon-k8s-dqlite is running
Service snap.microk8s.daemon-apiserver-kicker is running
Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
Copy processes list to the final report tarball
Copy disk usage information to the final report tarball
Copy memory usage information to the final report tarball
Copy server uptime to the final report tarball
Copy openSSL information to the final report tarball
Copy snap list to the final report tarball
Copy VM name (or none) to the final report tarball
Copy current linux distribution to the final report tarball
Copy asnycio usage and limits to the final report tarball
Copy inotify max_user_instances and max_user_watches to the final report tarball
Copy network configuration to the final report tarball
Inspecting kubernetes cluster
Inspect kubernetes cluster
Inspecting dqlite
Inspect dqlite
Building the report tarball
Report tarball is at /var/snap/microk8s/6541/inspection-report-20240326_172535.tar.gz
We have configured paths for the persistent volumes:
...
localPersistentVolumes:
enabled: true
node: "azuscesiumiondev01" # REQUIRED: Name returned by "kubectl get nodes"
# This will generate a PersistentVolumeClaim named "{{ .Release.Name }}-assets"
# Use that claim as a volume to store assets
assets:
enabled: true
path: "/mnt/cesium-data/cesium-assets" # REQUIRED: Path to output data on your filesystem.
capacity: 32Gi
# This will generate a PersistentVolumeClaim named "{{ .Release.Name }}-sources"
# Use that claim as a volume to store source data
sources:
enabled: true
path: "/mnt/cesium-data/cesium-source-data" # REQUIRED: Path to input data on your filesystem.
capacity: 32Gi
# This will generate a PersistentVolumeClaim named "{{ .Release.Name }}-stories"
# Use that claim as a volume to store story images
stories:
enabled: true
path: "/mnt/cesium-data/cesium-stories-images" # REQUIRED: Path to stories data on your filesystem.
capacity: 8Gi
# This will create a PersistentVolumeClaim named "{{ .Release.Name }}-postgresql"
# See the "postgresql" section below to configure this
postgresql:
enabled: true
path: "/mnt/cesium-data/cesium-postgresql" # REQUIRED: Path to postgres data on your filesystem.
capacity: 32Gi
...
The directories have been created on a secondary mounted drive:
:/$ ls -all /mnt/cesium-data/
total 40
drwxr-xr-x 7 serveradmin root 4096 Mar 26 15:49 .
drwxr-xr-x 3 root root 4096 Mar 26 14:43 ..
drwxrwxr-x 2 serveradmin serveradmin 4096 Mar 26 15:49 cesium-assets
drwxrwsr-x 3 serveradmin serveradmin 4096 Mar 26 15:51 cesium-postgresql
drwxrwxr-x 2 serveradmin serveradmin 4096 Mar 26 15:49 cesium-source-data
drwxrwxr-x 2 serveradmin serveradmin 4096 Mar 26 15:49 cesium-stories-images
drwx------ 2 root root 16384 Mar 26 13:53 lost+found
We have also tried to run with the debug flag, i.e.
install.go:178: [debug] Original chart version: ""
install.go:195: [debug] CHART PATH: /home/serveradmin/cesium-ion-1.0.0/cesium-ion
client.go:128: [debug] creating 1 resource(s)
client.go:229: [debug] checking 29 resources for changes
client.go:250: [debug] Created a new ServiceAccount called "cesium-ion" in cesium-ion
client.go:250: [debug] Created a new Secret called "cesium-ion-postgresql" in cesium-ion
client.go:512: [debug] Looks like there are no changes for Secret "cesium-ion-secrets"
client.go:250: [debug] Created a new Secret called "cesium-ion-tiling-secret" in cesium-ion
client.go:250: [debug] Created a new Secret called "cesium-ion-install-data-secret" in cesium-ion
client.go:250: [debug] Created a new ConfigMap called "cesium-ion-api-config-env" in cesium-ion
client.go:250: [debug] Created a new ConfigMap called "cesium-ion-api-config-file" in cesium-ion
client.go:250: [debug] Created a new ConfigMap called "cesium-ion-jobs" in cesium-ion
client.go:250: [debug] Created a new StorageClass called "cesium-ion-cesium-ion-local-storage" in
client.go:250: [debug] Created a new PersistentVolume called "cesium-ion-cesium-ion-assets-volume" in
client.go:250: [debug] Created a new PersistentVolume called "cesium-ion-cesium-ion-sources-volume" in
client.go:250: [debug] Created a new PersistentVolume called "cesium-ion-cesium-ion-postgresql-volume" in
client.go:250: [debug] Created a new PersistentVolume called "cesium-ion-cesium-ion-stories-volume" in
client.go:250: [debug] Created a new PersistentVolumeClaim called "cesium-ion-assets" in cesium-ion
client.go:250: [debug] Created a new PersistentVolumeClaim called "cesium-ion-sources" in cesium-ion
client.go:250: [debug] Created a new PersistentVolumeClaim called "cesium-ion-postgresql" in cesium-ion
client.go:250: [debug] Created a new PersistentVolumeClaim called "cesium-ion-stories" in cesium-ion
client.go:250: [debug] Created a new Role called "cesium-ion-launch-job" in cesium-ion
client.go:250: [debug] Created a new RoleBinding called "cesium-ion-launch-job-binding" in cesium-ion
client.go:250: [debug] Created a new Service called "cesium-ion-postgresql-hl" in cesium-ion
client.go:250: [debug] Created a new Service called "cesium-ion-postgresql" in cesium-ion
client.go:250: [debug] Created a new Service called "cesium-ion-assets" in cesium-ion
client.go:250: [debug] Created a new Service called "cesium-ion-api" in cesium-ion
client.go:250: [debug] Created a new Service called "cesium-ion-frontend" in cesium-ion
client.go:250: [debug] Created a new Deployment called "cesium-ion-api" in cesium-ion
client.go:250: [debug] Created a new Deployment called "cesium-ion-assets" in cesium-ion
client.go:250: [debug] Created a new Deployment called "cesium-ion-frontend" in cesium-ion
client.go:250: [debug] Created a new StatefulSet called "cesium-ion-postgresql" in cesium-ion
client.go:250: [debug] Created a new CronJob called "cesium-ion-maintainence" in cesium-ion
wait.go:48: [debug] beginning wait for 29 resources with timeout of 5m0s
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
ready.go:277: [debug] Deployment is not ready: cesium-ion/cesium-ion-api. 0 out of 1 expected pods are ready
Error: INSTALLATION FAILED: timed out waiting for the condition
helm.go:84: [debug] timed out waiting for the condition
INSTALLATION FAILED
main.newInstallCmd.func2
helm.sh/helm/v3/cmd/helm/install.go:127
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra@v1.4.0/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra@v1.4.0/command.go:974
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra@v1.4.0/command.go:902
main.main
helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
runtime/proc.go:250
runtime.goexit
runtime/asm_amd64.s:1598
Just from a quick glance, it looks like you have the Postgresql path as a /mnt/ path. That path must live inside of your WSL2 VM for example something like /home/{username}/cesiumion/postgresql for example. Also see below for the section of our help docs that mentions this specifically for WSL installs.
When editing cesium-ion/values.yaml on Windows, file paths should be from inside the WSL2 shell. For example, paths to your C:\ drive need to start with /mnt/c/ . Additionally, the postgresql volume must live inside of the WSL2 VM, for example at ~/ion/postgres , and can not reside under /mnt/c .
before you ran the helm install command after updating the path? This just makes sure there isn’t any old setting still around from previous installs. If you beat me too it and already did that, next thing is going to be to take a look at the logs for the pods. If you could attached those that would be great. The command to get them is here.
kubectl logs <POD-NAME>
If you could pull the logs for the postgres pod and the Api server pod that would really help.
serveradmin@azuscesiumiondev01:/mnt/cesium-data$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
container-registry registry-6c9fcc695f-pxrsl 1/1 Running 1 (41h ago) 42h
kube-system hostpath-provisioner-7df77bc496-bnp7k 1/1 Running 1 (41h ago) 42h
kube-system coredns-864597b5fd-fmttl 1/1 Running 3 (41h ago) 43h
kube-system calico-kube-controllers-77bd7c5b-8278k 1/1 Running 3 (41h ago) 43h
kube-system calico-node-c8cqw 1/1 Running 3 (41h ago) 43h
ingress nginx-ingress-microk8s-controller-g6vbj 1/1 Running 1 (41h ago) 42h
cesium-ion cesium-ion-api-86cc47c9df-4k7tc 0/1 Pending 0 8m53s
cesium-ion cesium-ion-assets-689d489f7-pjmhv 0/1 Pending 0 8m53s
cesium-ion cesium-ion-postgresql-0 1/1 Running 0 8m53s
cesium-ion cesium-ion-frontend-6464d78596-7nrbz 1/1 Running 0 8m53s
serveradmin@azuscesiumiondev01:/mnt/cesium-data$ kubectl --namespace cesium-ion logs cesium-ion-api-86cc47c9df-4k7tc
serveradmin@azuscesiumiondev01:/mnt/cesium-data$ kubectl --namespace cesium-ion logs cesium-ion-assets-689d489f7-pjmhv
serveradmin@azuscesiumiondev01:/mnt/cesium-data$ kubectl --namespace cesium-ion logs cesium-ion-postgresql-0
postgresql 09:38:00.95
postgresql 09:38:00.95 Welcome to the Bitnami postgresql container
postgresql 09:38:00.95 Subscribe to project updates by watching https://github.com/bitnami/containers
postgresql 09:38:00.95 Submit issues and feature requests at https://github.com/bitnami/containers/issues
postgresql 09:38:00.96
postgresql 09:38:00.97 INFO ==> ** Starting PostgreSQL setup **
postgresql 09:38:00.99 INFO ==> Validating settings in POSTGRESQL_* env vars..
postgresql 09:38:01.00 INFO ==> Loading custom pre-init scripts...
postgresql 09:38:01.00 INFO ==> Initializing PostgreSQL database...
postgresql 09:38:01.02 INFO ==> pg_hba.conf file not detected. Generating it...
postgresql 09:38:01.03 INFO ==> Generating local authentication configuration
postgresql 09:38:02.12 INFO ==> Starting PostgreSQL in background...
postgresql 09:38:02.24 INFO ==> Changing password of postgres
postgresql 09:38:02.26 INFO ==> Creating user db_user
postgresql 09:38:02.27 INFO ==> Granting access to "db_user" to the database "postgres"
postgresql 09:38:02.30 INFO ==> Setting ownership for the 'public' schema database "postgres" to "db_user"
postgresql 09:38:02.32 INFO ==> Configuring replication parameters
postgresql 09:38:02.34 INFO ==> Configuring synchronous_replication
postgresql 09:38:02.34 INFO ==> Configuring fsync
postgresql 09:38:02.38 INFO ==> Stopping PostgreSQL...
waiting for server to shut down.... done
server stopped
postgresql 09:38:02.49 INFO ==> Loading custom scripts...
postgresql 09:38:02.49 INFO ==> Enabling remote connections
postgresql 09:38:02.50 INFO ==> ** PostgreSQL setup finished! **
postgresql 09:38:02.51 INFO ==> ** Starting PostgreSQL **
2024-03-28 09:38:02.534 GMT [1] LOG: pgaudit extension initialized
2024-03-28 09:38:02.546 GMT [1] LOG: starting PostgreSQL 15.4 on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2024-03-28 09:38:02.547 GMT [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2024-03-28 09:38:02.547 GMT [1] LOG: listening on IPv6 address "::", port 5432
2024-03-28 09:38:02.554 GMT [1] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
2024-03-28 09:38:02.562 GMT [147] LOG: database system was shut down at 2024-03-28 09:38:02 GMT
2024-03-28 09:38:02.573 GMT [1] LOG: database system is ready to accept connections
2024-03-28 09:43:02.656 GMT [145] LOG: checkpoint starting: time
2024-03-28 09:43:06.696 GMT [145] LOG: checkpoint complete: wrote 43 buffers (0.3%); 0 WAL file(s) added, 0 removed, 0 recycled; write=4.018 s, sync=0.008 s, total=4.040 s; sync files=11, longest=0.004 s, average=0.001 s; distance=253 kB, estimate=253 kB
Seems like it’s the API and Assets pods that aren’t running but I’m not getting any logs out of them.
I then ran a describe on the two pods, and it seems to be complaining about insufficient CPU… which is odd because we have 4 cores as per the requirements in the documentation.
serveradmin@azuscesiumiondev01:/mnt/cesium-data$ k describe pod cesium-ion-api-ddbddcd78-sb54s
Name: cesium-ion-api-ddbddcd78-sb54s
Namespace: cesium-ion
Priority: 0
Service Account: cesium-ion
Node: <none>
Labels: app.kubernetes.io/instance=cesium-ion
app.kubernetes.io/name=cesium-ion
cesiumionPurpose=api-server
pod-template-hash=ddbddcd78
Annotations: checksum/config: de6d1e354651a8c8272f7020a41a995eb3086a5768ee8b1cb041097e8ca1f983
Status: Pending
IP:
IPs: <none>
Controlled By: ReplicaSet/cesium-ion-api-ddbddcd78
Containers:
cesium-ion-api:
Image: localhost:32000/cesium-ion:1.0.0
Port: 8081/TCP
Host Port: 0/TCP
Args:
self-hosted-entrypoint
Limits:
cpu: 4
memory: 1Gi
Requests:
cpu: 4
memory: 1Gi
Liveness: http-get http://:http/health delay=30s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get http://:http/health delay=30s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
cesium-ion-api-config-env ConfigMap Optional: false
cesium-ion-secrets Secret Optional: false
Environment: <none>
Mounts:
/data/assets from assets-volume (rw)
/data/sources from sources-volume (rw)
/data/stories from stories-volume (rw)
/tmp from temp-directory (rw)
/var/app/packages/ion-api/config.json from config (ro,path="config.json")
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bczws (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
assets-volume:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: cesium-ion-assets
ReadOnly: false
sources-volume:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: cesium-ion-sources
ReadOnly: false
stories-volume:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: cesium-ion-stories
ReadOnly: false
temp-directory:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: 1Gi
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: cesium-ion-api-config-file
Optional: false
kube-api-access-bczws:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Guaranteed
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 2m54s default-scheduler 0/1 nodes are available: 1 Insufficient cpu. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod..
Warning FailedScheduling 8m4s (x2 over 8m6s) default-scheduler 0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
serveradmin@azuscesiumiondev01:/mnt/cesium-data$ k describe pod cesium-ion-assets-694585f664-jkwmx
Name: cesium-ion-assets-694585f664-jkwmx
Namespace: cesium-ion
Priority: 0
Service Account: cesium-ion
Node: <none>
Labels: app.kubernetes.io/instance=cesium-ion
app.kubernetes.io/name=cesium-ion
cesiumionPurpose=asset-server
pod-template-hash=694585f664
Annotations: checksum/config: d721a9511fc50062e160848290d2dc908f973c790a11da9967675b046b5b54e6
Status: Pending
IP:
IPs: <none>
Controlled By: ReplicaSet/cesium-ion-assets-694585f664
Containers:
cesium-ion-assets:
Image: localhost:32000/cesium-ion-asset-server:1.0.0
Port: 8070/TCP
Host Port: 0/TCP
Args:
--directory
/data/assets
--concurrency
8
--ion
--storiesDir
/data/stories
--cors
Limits:
cpu: 4
memory: 4Gi
Requests:
cpu: 4
memory: 4Gi
Liveness: http-get http://:http/health delay=10s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get http://:http/health delay=10s timeout=1s period=10s #success=1 #failure=3
Environment:
CESIUM_LICENSE: <set to the key 'CESIUM_LICENSE' in secret 'cesium-ion-secrets'> Optional: false
ASSET_TOKEN_SECRET: <set to the key 'tokenAuthentication__assets_secret' in secret 'cesium-ion-secrets'> Optional: false
Mounts:
/data/assets from assets-volume (rw)
/data/stories from stories-volume (rw)
/tmp from temp-directory (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-rcshq (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
assets-volume:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: cesium-ion-assets
ReadOnly: false
stories-volume:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: cesium-ion-stories
ReadOnly: false
temp-directory:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: 1Gi
kube-api-access-rcshq:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Guaranteed
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 3m9s default-scheduler 0/1 nodes are available: 1 Insufficient cpu. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod..
Warning FailedScheduling 8m18s (x2 over 8m20s) default-scheduler 0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
serveradmin@azuscesiumiondev01:/mnt/cesium-data$
Ok awesome, I’m glad you were able to get it up and running. It was definitely a resource allocation issue then based on the logs you attached and the fact that scaling those CPU values down got the cluster up and running.
It is totally acceptable to drop the requested CPU values in the .yaml file to ‘1’. It just might mean that there are cases in a workflow that some CPU bottlenecks are observed. This is highly dependent on the workflow and expected usage though. If you do notice that it is slower than you or the users would like, you can just increase the number of cores that the VM has allocated to it. You would then also need to adjust those CPU values in the .yaml file to take into account the extra cores in the VM. The default values currently in the .yaml file are our “recommended” numbers that might need to be adjusted depending on system specifications. I am going to add a note to our docs to make that clear going forward, especially since it does not align with the “minimum” specs we have listed.
Let me know if you have any issues or have any other questions.