Skip to main content

 Error "Your subscription has expired. The core product operations are unavailable" is seen after upgrading to Aria Automation/Aria Orchestrator 8.18.x

https://knowledge.broadcom.com/external/article?articleNumber=380260

Issue/Introduction

After upgrading to 8.18.x we see the following error when trying to run actions or workflows in Aria Orchestrator.

Error: "Your subscription has expired. The core product operations are unavailable. Please contact your Account Manager or Customer Success Manager to renew your license".

Below events can be seen in vco-server-app.log 

2024-07-22T04:03:42.697Z WARN vco [host='vco-app-##########-#####' thread='licenseComplianceScheduler-1' user='' org='' trace=''] {} com.vmware.o11n.service.license.retriever.LicenseComplianceRetriever - Error occurred during license compliance retrieval.
com.vmware.o11n.service.license.RetrieveLicenseException: Request to automation license endpoint failed.

2024-07-22T04:03:42.701Z WARN vco [host='vco-app-##########-#####' thread='licenseComplianceScheduler-1' user='' org='' trace=''] {} com.vmware.o11n.service.license.task.LicenseComplianceTask - License compliance state is restricted. 

2024-07-22T04:08:53.369Z ERROR vco [host='vco-app-##########-#####' thread='http-nio-8280-exec-118' user='' org='' trace='-'] {} com.vmware.o11n.web.common.BaseController - Error invoking REST [unknown]
ch.dunes.util.DunesServerException: ch.dunes.util.OperationNotEntitledException: Requested operation is not permitted due to no valid license entitlement

Environment

VMware Aria Orchestrator 8.18.x

VMware Aria Automation 8.18.x

Cause

The observed failures happen intermittently. 

Some calls to the orchestrator are failing with errors after the license entitlement restriction checks is reported as restricted. The license check in turn has failed due to usage of an expired token used for authentication. After several minutes the expired token is refreshed and the license restriction is lifted resulting in continuing normal operation.

Resolution

  • Run the below command on any one node within the cluster to apply the 'license check system property' and wait for the vco pods to restart.

    # vracli vro properties set --key com.vmware.o11n.license.check.automation-endpoint.enabled --value false
    • For the embedded Orchestrator this command is run on the Aria Automation Node.

Additional Information

Comments

Popular posts from this blog

Step-by-Step Explanation of Ballooning, Compression & Swapping in VMware

 🔹 Step-by-Step Explanation of Ballooning, Compression & Swapping in VMware ⸻ 1️⃣ Memory Ballooning (vmmemctl) Ballooning is the first memory reclamation technique used when ESXi detects memory pressure. ➤ Step-by-Step: How Ballooning Works  1. VMware Tools installs the balloon driver (vmmemctl) inside the guest OS.  2. ESXi detects low free memory on the host.  3. ESXi inflates the balloon in selected VMs.  4. Balloon driver occupies guest memory, making the OS think RAM is full.  5. Guest OS frees idle / unused pages (because it believes memory is needed).  6. ESXi reclaims those freed pages and makes them available to other VMs. Why Ballooning Happens?  • Host free memory is very low.  • ESXi wants the VM to release unused pages before resorting to swapping. Example  • Host memory: 64 GB  • VMs used: 62 GB  • Free: 2 GB → ESXi triggers ballooning  • VM1 (8 GB RAM): Balloon inflates to 2 GB → OS frees 2 GB → ESXi re...

ESXi Host Troubleshooting Checklist

  🛠️ ESXi Host Troubleshooting Checklist (With Complete Log Locations) ✅ 1. Host Status & Connectivity Check host state in vCenter (Connected / Not Responding / Disconnected) Review CPU, RAM, and datastore usage Validate HA/DRS recommendations ✅ 2. Hardware Health Monitor hardware sensors (CPU, DIMMs, fans, PSU, RAID) Check RAID controller logs via vendor tools View hardware status in: vCenter → Monitor → Hardware Health DCUI → Hardware Status ✅ 3. Network Validation Verify vSwitches, Port Groups, NIC teaming, VLAN tagging Check vmkernel ports (mgmt, vMotion, iSCSI, vSAN) Test reachability using: Shell vmkping < IP > Show more lines Validate physical NIC status and link speed ✅ 4. Storage & Datastore Checks Confirm datastore accessibility Rescan storage adapters (iSCSI/FC/NFS) Validate multipathing (Round Robin / Fixed / MRU) Check datastore latency with esxtop ✅ **5. Critical Logs & Their Locations (Full List) Here are key ESXi log files that every VMware adm...
  vCenter Troubleshooting Tips, Common Issues & Log Locations 🚀 1. General Troubleshooting Approach A. Check service health first For vCenter Server Appliance (VCSA): Shell https : / / < vcenter - FQDN > : 5480 Show more lines Go to Services → Health to verify: vCenter Server ESXi hosts vSphere Client DNS, NTP, DB connections vpxd, vpxd-svcs, vsphere-ui B. DNS & NTP checks vCenter is highly dependent on correct DNS & time sync. Check: Shell nslookup < vcenter - fqdn > nslookup < esxi - host - fqdn > Show more lines Time drift > 5 minutes causes: Login failures vpxd crashes PSC authentication issues C. Check storage/database health Slow DB/storage = UI slow, tasks hung. For VCSA: Shell df - h du - sh / storage / log / * / var / log Show more lines Free up space if partitions hit >80%. D. Restart critical services safely On VCSA: Shell service - control - - status service - control - - stop - - all service - control - - start - - all ...