I am trying to deploy MES 11.1.0 into AKS and am receiving a 404 error right after deploying lbogenerator-auth. I am not sure what the install scripts are trying to access that is giving a 404, or where to look. Is there any way to turn on more verbose logging or gain any insights into what exactly is failing? The installer can connect to my cluster, as other pods/secrets/config maps are deployed properly.
Thanks,
Matt
Hi Matt,
Cannot really understand where did you get the 404 error. Did you saw on the log files? Did you tried to access the UI and got a 404?
The deployment of lbogenerator-auth strongly indicates a communication or access problem within your Azure Kubernetes Service (AKS) environment. The lbogenerator-auth serves as middleware for a traefik-forwardauth pod (if i am not wrong), which typically manages authentication forwarding when accessing MES components.
Pinpointing the precise cause of the 404 without additional information is difficult. To effectively troubleshoot this, let’s gather more specific details.
Clarification Required for Diagnosis
To facilitate an accurate diagnosis of the issue, please provide clarification on the following points:
404 Error Context: What specific resource or endpoint is generating the 404 error? Is it an external URL, an internal service within the cluster, or another type of resource?
Please investigate the following areas within your environment:
AKS Resource Verification: Given that the installer successfully connects and deploys other resources, it is crucial to verify the operational status of your MES deployments and pods (https://help.criticalmanufacturing.com/operationguide/system-administration/container-stack/).
CrashLoopBackOff state, Pending status, or other error conditions within their respective logs.Examine the logs of the lbogenerator-auth pod, as well as any associated traefik or forwardauth pods, for more detailed error messages. These logs are likely to contain more verbose diagnostic information.
Customer Portal Connectivity: MES installations frequently require outbound network access to Critical Manufacturing’s customer portals for licensing and other essential services. Ensure that your AKS environment possesses the necessary outbound connectivity according to following guide: https://portal.criticalmanufacturing.com/Help/devops-center/deployment-targets/
Once we have a clearer understanding of the specific context of the 404 error and the operational status of your pods, we can proceed with a more targeted investigation of potential causes.
Best regards,
João Moreira
Hi João,Thanks for getting back to me. The only pods that are deployed currently are ‘lbogenerator’, ‘mcad’, and ‘mes-scheduler’. The ‘mes-scheduler’ is running properly, however the other two are unhealthy with the error: Readiness probe failed: Get “http://10.244.1.135:8080/Api/Ping”: dial tcp 10.244.1.135:8080: connect: connection refused. This makes sense, as the pods aren’t ‘ready’ and are therefore the probe isnt returning. Doing the ‘kubectl logs -f’ command reveals ‘Info | Waiting for Environment Manager’.My guess is that the pods are waiting for something else to come online, but nothing else was deployed in the cluster. As far as I can tell, there is nothing configured that would block outbound internet access to the links specified in the deployment-targets page. ‘lbogenerator-auth’ pod does not exist, as the installation page fails during this deploy step with the above ‘404’ error.
I forgot to answer the first part of your message. Currently the Installation aborts, and its the Customer Portal itself that is returning the 404: “Error while interacting with the Kubernetes Cluster: Operation returned an invalid status code ‘NotFound’, response body 404 page not found.” There is no other context provided, nothing in any of the pod logs point to a 404, and since the error is coming from the Customer Portal, I can’t find a way to enable more logging.