Troubleshooting vRealize IaaS SSL Certificate Registration Issue


Today I was working on replacing SSL certificates on my vRealize IaaS server so as to complete the SSL certificate replacement on all vRealize components that I have deployed in my lab.

I am not going to cover here the steps needed to generate signed SSL certificates as I have covered them in my earlier  blog posts. If you are looking for Step by Step demonstration of the same please have a look on this Article Grant Orchard.

After replacing the IaaS SSL certificates, IaaS Server needs to re-register the new certificates to the vRA Appliance. The commands used to achieve this is as follows:

# cd C:\Program Files (x86)\VMware\vCAC\Server\Model Manager Data\Cafe

# vcac-config RegisterEndpoint –EndpointAddress https://vra1.alex.local/vcac –Endpoint ui -v

# vcac-config RegisterEndpoint –EndpointAddress https://vra1.alex.local/vcac/Repository –Endpoint repo -v

# vcac-config RegisterEndpoint –EndpointAddress https://vra1.alex.local/vcac/WAPI –Endpoint wapi -v

# vcac-config RegisterEndpoint –EndpointAddress https://vra1.alex.local/vcac/WAPI/api/status –Endpoint status -v

Note: vra1.alex.local is the name of my vRA appliance. Replace this with your appliance FQDN.

When I ran the above commands all of them was failing with below error messages.

Note: I have not included the full error message and only included the part which were making some sense (in Italics) as its too long to paste entire error message here.

System.Data.Services.Client.DataServiceQueryException: An error occurred while processing this request. —> System.Data.Services.Client.DataServiceClientException: <!DOCTYPE html>
<html>
<head>
<title>The system cannot find the file specified</title>
<meta name=”viewport” content=”width=device-width” />
<style>
body {font-family:”Verdana”;font-weight:normal;font-size: .7em;color:black;}
p {font-family:”Verdana”;font-weight:normal;color:black;margin-top: -5px}
b {font-family:”Verdana”;font-weight:bold;color:black;margin-top: -5px}
H1 { font-family:”Verdana”;font-weight:normal;font-size:18pt;color:red }
H2 { font-family:”Verdana”;font-weight:normal;font-size:14pt;color:maroon }
pre {font-family:”Consolas”,”Lucida Console”,Monospace;font-size:11pt;margin:0;padding:0.5em;line-height:14pt}
.marker {font-weight: bold; color: black;text-decoration: none;}

<b> Description: </b>An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

<br><br>

<b> Exception Details: </b>System.ComponentModel.Win32Exception: The system cannot find the file specified<br><br>

[SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 – Could not open a connection to SQL Server)]

System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction) +6675286

System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover) +219
System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout) +6703968

System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance) +6704427

As you can see from above error message that something was wrong with my SQL Server. I was scratching my head what could have gone wrong. I decided to test my SQL server connectivity first.

I launched SQL management studio and tried connecting to the vRA-IaaS DB instance using ‘sa’ user. It failed and I was wondering why. Then I decided to give a shot to the windows authentication method (as sometimes my ‘sa’ user have thrown tantrum at me)

To my surprise windows authentication was also not working.

iaasssl-1

Then I checked whether or not SQL services are running (of course this I should have checked at first place) and found SQL services were dead.

iaasssl-2

I understood what was wrong. Before replacing the SSL certs when I tried login to IaaS server using the service account ‘svcvcac’ as visible in above screenshots, Windows was telling me that password has expired and needs to be changed.

I forgot that I have configured the same service account to be used in my SQL server (I should have used a separate dedicated one for sql) and due to password change SQL services died as SQL is using this service account to run as.

I changed the password to original password (at the time of IaaS installation) and restarted my SQL services which started without any issues.

Now it was time to re-run the certificate re-register commands. This time all the commands completed successfully.

iaasssl-3

Lessons learnt from this mistake can be summarized as:

1: Service accounts should have “Password Never Expire” or longer duration for password expiry then any other normal domain user accounts

2: Use a dedicated service accounts for SQL servers.

Again I would say mistakes are essential for us to teach us really good lessons.

I hope you enjoyed reading this post. Feel free to share this on social media if it is worth sharing. Be sociable 🙂

About Alex Hunt

Hi All I am Manish Kumar Jha aka Alex Hunt. I am currently working in VMware Software India Pvt Ltd as Operations System Engineer (vCloud Air Operations). I have around 5 Years of IT experience and have exposure on VMware vSphere, vCloud Director, RHEL and modern data center technologies like Cisco UCS and Cisco Nexus 1000v and NSX. If you find any post informational to you please press like and share it across social media and leave your comments if you want to discuss further on any post. Disclaimer: All the information on this website is published in good faith and for general information purpose only. I don’t make any warranties about the completeness, reliability and accuracy of this information. Any action you take upon the information you find on this blog is strictly at your own risk. The Views and opinions published on this blog are my own and not the opinions of my employer or any of the vendors of the product discussed.
This entry was posted in Vmware, vRealize Automation. Bookmark the permalink.

One Response to Troubleshooting vRealize IaaS SSL Certificate Registration Issue

  1. Pingback: Newsletter: November 21, 2015 | Notes from MWhite

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s