Skip to main content

 

New installs of vRealize Orchestrator 8.x fail to install due to a POD STATUS of 'CrashLoopBackOff'


Issue/Introduction

Symptoms:

When deploying a new installation of vRealize Orchestrator 8.x, or vRealize Automation 8.x with embedded vRealize Orchestrator 8.x, you may experience the following:

  • vRealize Suite Lifecycle Manager displays a failure to initialize the VRA cluster with the error LCMVRAVACONFIG590003.

  • The Kubernetes pod for vco-app-<ID> fails to start with a STATUS of 'CrashLoopBackOff'.  To confirm this, run the following command on the appliance:

kubectl -n prelude get pods

  • The status of the vco-app pod is CrashLoopBackOff

  • Confirm the error by running the following command on the appliance:

kubectl -n prelude logs vco-app-<ID from previous command> -c install-rpms

  • The vco-app-<ID> container log contains the following error:

You are required to change your password immediately (password expired)
groupadd: PAM: Authentication token is no longer valid; new one required
useradd: group 'vco' does not exist
error: %prein(vco-server-8.1.0.1576591045-15331417.noarch) scriptlet failed, exit status 6
error: vco-server-8.1.0.1576591045-15331417.noarch: install failed

Environment

VMware vRealize Automation 8.x
VMware vRealize Orchestrator 8.x

Cause

The deployed vRealize Orchestrator container may be created with an expired password causing packages to fail and the container to crash.

Resolution

Workaround:
To work around this issue, use the following steps:

  1. Log into the Appliance with SSH.
  2. Navigate to /opt/charts/vco/templates/
  3. Copy/Backup the deployment.yaml file using the command:
cp deployment.yaml /tmp/

NOTE:  Do not copy any backup.yaml files into /opt/charts/vco/ or /opt/charts/vco/templates/.  Doing so will result in 404 errors after successful service restart.
  1. Edit the deployment.yaml file using your preferred editor (vi deployment.yaml).
  2. Locate the string:  init_run.sh
  3. Replace the text as follows.
Search for row that contains this text: init_run.sh:
command:
- "/bin/bash"
- "-c"
- "/init_run.sh"

Edit this row by just adding these two "sed" commands before /init_run.sh script. The row after editing should looks similar to this::
command:

- "/bin/bash"
- "-c"
- "sed -i 's/root:.*/root:x:18135:0:99999:7:::/g' /etc/shadow && sed -i 's/vco:.*/vco:x:18135:0:99999:7:::/g' /etc/shadow && /init_run.sh"
  1. Save the file
If you are deploying outside of vRealize Suite Lifecycle Manager, follow these steps:
  1. Navigate to /opt/scripts/
  2. Execute the script: ./deploy.sh
If you are deploying through vRealize Suite Lifecycle Manager, follow these steps:
  1. Log back into Lifecycle Manager and 'Retry' the deployment that failed.  Installation should now progress as expected.
Labels:

Comments

Post a Comment

Popular posts from this blog

  Issue with Aria Automation Custom form Multi Value Picker and Data Grid https://knowledge.broadcom.com/external/article?articleNumber=345960 Products VMware Aria Suite Issue/Introduction Symptoms: Getting  error " Expected Type String but was Object ", w hen trying to use Complex Types in MultiValue Picker on the Aria for Automation Custom Form. Environment VMware vRealize Automation 8.x Cause This issue has been identified where the problem appears when a single column Multi Value Picker or Data Grid is used. Resolution This is a known issue. There is a workaround.  Workaround: As a workaround, try adding one empty column in the Multivalue picker without filling the options. So we can add one more column without filling the value which will be hidden(there is a button in the designer page that will hide the column). This way the end user will receive the same view.  
Post a Comment
Read more

57 Tips Every Admin Should Know

Active Directory 1. To quickly list all the groups in your domain, with members, run this command: dsquery group -limit 0 | dsget group -members –expand 2. To find all users whose accounts are set to have a non-expiring password, run this command: dsquery * domainroot -filter “(&(objectcategory=person)(objectclass=user)(lockoutTime=*))” -limit 0 3. To list all the FSMO role holders in your forest, run this command: netdom query fsmo 4. To refresh group policy settings, run this command: gpupdate 5. To check Active Directory replication on a domain controller, run this command: repadmin /replsummary 6. To force replication from a domain controller without having to go through to Active Directory Sites and Services, run this command: repadmin /syncall 7. To see what server authenticated you (or if you logged on with cached credentials) you can run either of these commands: set l echo %logonserver% 8. To see what account you are logged on as, run this command: ...
Post a Comment
Read more
  The Guardrails of Automation VMware Cloud Foundation (VCF) 9.0 has redefined private cloud automation. With full-stack automation powered by Ansible and orchestrated through vRealize Orchestrator (vRO), and version-controlled deployments driven by GitOps and CI/CD pipelines, teams can build infrastructure faster than ever. But automation without guardrails is a recipe for risk Enter RBAC and policy enforcement. This third and final installment in our automation series focuses on how to secure and govern multi-tenant environments in VCF 9.0 with role-based access control (RBAC) and layered identity management. VCF’s IAM Foundation VCF 9.x integrates tightly with enterprise identity providers, enabling organizations to define and assign roles using existing Active Directory (AD) groups. With its persona-based access model, administrators can enforce strict boundaries across compute, storage, and networking resources: Personas : Global Admin, Tenant Admin, Contributor, Viewer Projec...
Post a Comment
Read more