Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
New Member

ViRL/CML simulation issues "Failed loading nodes from Open Stack"

Hello,

 

Currently testing ViRL ( not sure what the actual version is) on an ESX 5.5i server host. Running simulation on an Eclipse (VMaestro) client  PC  connected to the ViRL server results in the following:

1. Failed loading nodes from OpenStack: HTTP-status: "500", HTTP-reason: "Internal Server Error", response: "{"computeFault": {"message": "The server has either erred  or is incapable of performing the requested operation.", "code": 500}}"

2. The User Work space management console ( Admin mode), confirms that are no Open stack nodes found. All settings in this console display ACTIVE status for  nodes, networks, projects etc. 

3. All the web services on the Eclipse (VMaestro) client - Roster, Simulation Engine, OpenStack Autonetkit -  connected to the ViRL server show GREEN status as COMPATIBLE.

4. Executing Restack script from CLI (devstack directory) , results in an error and crashes the ViRL and the ViRL VM has to be reinstalled.

Thanks and Regards.

 

 

 

 

 

3 ACCEPTED SOLUTIONS

Accepted Solutions
Cisco Employee

Hello,Could you please open a

Hello,

Could you please open a TAC case for this issue.

Thanks and Regards,

Bikash

Cisco Employee

Hi Mohan,The VIRL version is

Hi Mohan,

The VIRL version is not TAC supported.

Please get the new OVA , deploy  it and let us know if the issue persists.

Thanks and Regards,

Bikash

Cisco Employee

Hello Mohan,Please find the

Hello Mohan,

Please find the answers below to your query.

1.Whether we require FLAT ( or SNAT ) for this Orchestration or is it purely related to OOB access to the routers bypassing VMaestro?

It is not purely related to OOB access, you can do external connectivity to the routers running in the simulation by using following ways:

Set Up a FLAT Network for Out of Band (OOB) Management Access
Connecting to external devices using FLAT Inband access
Connecting to external devices using SNAT Inband access


2.Does the IP pool for this FLAT network is assigned from an initialization file *.ini for the 172.16.1.0 pool?

Yes you are correct.

3.Which is the Management network required for UWM orchestration as far as ESX port groups are concerned?

It takes the eth0 IP as the management Network required for UWM orchestration.

Thanks and Regards,

Bikash

 

 

16 REPLIES
Cisco Employee

Hello,Thanks for posting the

Hello,

Thanks for posting the issue.

Kindly share the output of $virl_health_status  and below logs from CML server to analyze the issue.
$var/local/virl/logs
$var/log/syslog

Also share the screenshot of the issue and generate problem report from CML client.

Steps for generating problem report: Go to CML client->Navigate to help->Generate Problem Report.

Thanks and Regards,

Bikash

New Member

 Hi Bikash, Thank you for

 

Hi Bikash,

 

Thank you for your quick response and here are the captures, logfiles and CML client problem reports.

Please let me know if you need any information.

 

Thanks and Regards,

Mohan

Cisco Employee

Hello,From the $virl_health

Hello,

From the $virl_health_status I could see all the openstack services are not available, which leads the issue.

OpenStack image service for STD is not available
OpenStack compute service for STD is not available
OpenStack network service for STD is not available
OpenStack identity service for STD is not available  

Please let us know you got this issue in newly installed CML or it was working earlier and now you are facing the issue.

If it is on newly installed server then there may be some installation problem.

If it was working earlier,please let me know whether you have made any changes in to it.

Also please share output of below commands on CML server.

$nova service-list

$keystone user-list

Thanks and Regards,

Bikash

 

New Member

Hi Bikash,Thanks very much

Hi Bikash,

Thanks very much for the update.

I noticed it too but the message on Workspace Management - "User "uwmadmin" has no STD endpoint and cannot run simulations." - i thought is only applicable to the uwmadmin user and not too other users like guest.

The ViRL was never working (meaning Open stack node Internal error 500 ) and this is a new install. but on the client side, all the services are working including connectivity to the Server.

virl@virldu:/var$ nova service-list
ERROR: Cannot connect to proxy. Socket error: [Errno -2] Name or service not known.
virl@virldu:/var$ keystone user-list
WARNING: Bypassing authentication using a token & endpoint (authentication credentials are being ignored).
Unable to establish connection to http://localhost:35357/v2.0/users
virl@virldu:/var$ 

installation wise, i just installed the ViRL OVA on ESX 5.5 host and there were two other files i think one was with *. mf and *.vmdk extensions.

New Member

Just to add to the above,

Just to add to the above, prior to ViRL installation i had created two networks FLAT and SNAT on ESX, but during installation only Management Interfaces appear.

Thanks and Regards.

Cisco Employee

Hello,Could you please open a

Hello,

Could you please open a TAC case for this issue.

Thanks and Regards,

Bikash

New Member

OK, Bikash, will do. I

OK, Bikash, will do. I noticed that the OVA image size is only 11KB, whereas this should be around 4GB. So will try to download the image again and install and then raise TAC if required.  By the way, is it possible to get the procedure how to install CML *.vmdk  files on ESX.

 

Thanks and Regards,

Mohan

Cisco Employee

Hello Mohan,Thanks for the

Hello Mohan,

Thanks for the update.

Please re install and let us know if you are facing any issues. Open a TAC case if the issue persists.

For uploading .vmdk files to CML server, please copy the image using SFTP to CML server and try to load the image from UWM with using local URL path  (where the image is copied ex: home/virl/<filename>).

Thanks and Regards,

Bikash

New Member

 Thanks will raise a TAC case

 

Thanks will raise a TAC case as the fresh install from OVA appears to run into the same issue. I managed to convert ovf to ova using the vmware ovftool, but the problem still persists.

Also tried to install *.vmdk image on UWM, but get an error message if  the local URL path is provided ( the image was copied to /home/virl via sftp) and i have tried with the "server" subtype for this. Do you need a different subtype for uploading vmdk or can choose any of the available subtypes here?

Thanks again very much for your help.

Mohan

 

 

Cisco Employee

Hello Mohan,Thanks for the

Hello Mohan,

Thanks for the update.

The download should appear as ESXi specific OVA file, so you don't need any conversion and directly deploy that OVA to ESXi.

Please let me know from where you are downloading the file and for uploading image the above mentioned process should work.

 

Thanks and Regards,

Bikash

New Member

Hi Bikash,Sorry for late

Hi Bikash,

Sorry for late reply. the image was provided by our clients and this contained *.mf, *.vmdk and *.ovf files. 

1. Installation was tried with deploying the ovf file on ESXi which worked fine but simulation issues due to OpenStack was the problem.

2. Converted ovf to ova using the vmware ovftool and installed OVA with the same results as above. 

3. I am not sure if the ViRL version is TAC supported, so have requested for a new OVA version .So will let you know how we go. 

Thanks and Regards

 

 

Cisco Employee

Hi Mohan,The VIRL version is

Hi Mohan,

The VIRL version is not TAC supported.

Please get the new OVA , deploy  it and let us know if the issue persists.

Thanks and Regards,

Bikash

New Member

Hi Bikas,Thank you and will

Hi Bikas,

Thank you and will update as soon as i get and deploy the new OVA file for ViRL. 

In the meantime, i am trying to understand how the vIOS routers are orchestrated from UWM and interactions among AutonetKit, L2 Bridge and UWM orchestration. Please find attached a sketch below of the VirL  network that has been set up in our lab, brief description is as follows:

Virl Server is deployed within ESX5i server. The server has two physical interfaces eth0 and eth1. Two port groups one for (ViRL server access) and another for Flat network is created on ESX attached to the same vmnic0. The ViRL server is accessed from an external laptop running VMaestro and defined with new session profile to communicate to the four web services.

Going through the CML documentation, understand that two physical interfaces are required a) for Server access (in this case will be via VLAN 1) and b) FLAT network on VLAN 597 - I am assuming that this is for OOB access to the routers or Management network required for Orchestration of vIOS routers via UWM??

From the UWM it was observed that the vIOS routers receive                      IP addresses  from the 10.11.12.0/24 pool when Autonet kit is           instantiated from the ViRL client.

Now, what i fail is to understand (please correct me if i am wrong) is:

1) whether we require FLAT ( or SNAT ) for this Orchestration or is it purely related to OOB access to the routers bypassing VMaestro?

2) Does the IP pool for this FLAT network is assigned from an initialization file *.ini for the 172.16.1.0 pool.

3) Which is the Management network required for UWM orchestration as far as ESX port groups are concerned?

 

Thanks and Regards,

Mohan

Cisco Employee

Hello Mohan,Please find the

Hello Mohan,

Please find the answers below to your query.

1.Whether we require FLAT ( or SNAT ) for this Orchestration or is it purely related to OOB access to the routers bypassing VMaestro?

It is not purely related to OOB access, you can do external connectivity to the routers running in the simulation by using following ways:

Set Up a FLAT Network for Out of Band (OOB) Management Access
Connecting to external devices using FLAT Inband access
Connecting to external devices using SNAT Inband access


2.Does the IP pool for this FLAT network is assigned from an initialization file *.ini for the 172.16.1.0 pool?

Yes you are correct.

3.Which is the Management network required for UWM orchestration as far as ESX port groups are concerned?

It takes the eth0 IP as the management Network required for UWM orchestration.

Thanks and Regards,

Bikash

 

 

New Member

Hi Bikash,Installed the new

Hi Bikash,

Installed the new OVA and finally got it working!!!! Thank you very much for your support..i guess the previous image was corrupt and that was the reason we ran into the Open Stack issues.

 

virl@virl:~$ virl_health_status
Disk usage:
Filesystem                 Size  Used Avail Use% Mounted on
/dev/mapper/virl--vg-root   31G  6.8G   23G  24% /
udev                       3.9G  4.0K  3.9G   1% /dev
tmpfs                      798M  436K  798M   1% /run
none                       5.0M     0  5.0M   0% /run/lock
none                       3.9G   92K  3.9G   1% /run/shm
cgroup                     3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/sda1                  228M   57M  159M  27% /boot

MySQL is available

RabbitMQ status:
[{pid,3726},
 {running_applications,[{rabbit,"RabbitMQ","2.7.1"},
                        {os_mon,"CPO  CXC 138 46","2.2.7"},
                        {sasl,"SASL  CXC 138 11","2.1.10"},
                        {mnesia,"MNESIA  CXC 138 12","4.5"},
                        {stdlib,"ERTS  CXC 138 10","1.17.5"},
                        {kernel,"ERTS  CXC 138 10","2.14.5"}]},
 {os,{unix,linux}},
 {erlang_version,"Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:30] [kernel-poll:true]\n"},
 {memory,[{total,27973176},
          {processes,12612600},
          {processes_used,12600232},
          {system,15360576},
          {atom,1124441},
          {atom_used,1120270},
          {binary,192768},
          {code,11134417},
          {ets,921680}]},
 {vm_memory_high_watermark,0.39999999992828933},
 {vm_memory_limit,3346782617}]
RabbitMQ configured for OpenStack services is available

OpenStack image service for STD is available
OpenStack compute service for STD is available
OpenStack network service for STD is available
OpenStack identity service for STD is available

STD server configuration:
VIRL environment priority (lowest->highest): global conf, local conf, SHELL env, CLI args
Global config can be defined at "/etc/virl/virl.cfg"
Local config can be defined at "/home/virl/virl.cfg"
To set as SHELL ENV var: export NAME=value
To unset as SHELL ENV var: unset NAME
=========================================================
Your global config:
VIRL_STD_HOST = 0.0.0.0
VIRL_STD_PORT = 8080
VIRL_STD_PROCESS_COUNT = 5
VIRL_STD_USER_NAME = guest
VIRL_STD_DIR = /var/local/virl
=========================================================
Your local config:
=========================================================
Your SHELL environment:
=========================================================
Highest priority:
VIRL_STD_HOST = 0.0.0.0
VIRL_STD_PORT = 8080
VIRL_STD_PROCESS_COUNT = 5
VIRL_STD_USER_NAME = guest
VIRL_STD_DIR = /var/local/virl
=========================================================
STD/UWM user list read failed. Please check if the STD/UWM database is initialized and readable by the current user.
STD server on url http://localhost:8080 is listening
UWM server on url http://localhost:8081 is listening

STD server version:
{u'client-compatible': False,
 u'distro-version': u'Ubuntu 12.04.4 LTS',
 u'features': [u'no jobs',
               u'subtypes',
               u'licensing',
               u'systemlogs',
               u'list',
               u'status:nodes',
               u'export',
               u'export:updated-addresses',
               u'export:updated-startup-configs',
               u'export:running-configs',
               u'export:startup-configs',
               u'update:start',
               u'update:stop',
               u'vnc-console',
               u'serial-port'],
 u'kernel-version': u'3.8.0-41-generic x86_64',
 u'kvm-ok': u'INFO: /dev/kvm exists\nKVM acceleration can be used',
 u'kvm-version': u'QEMU emulator version 1.0 (qemu-kvm-1.0), Copyright (c) 2003-2008 Fabrice Bellard',
 u'openstack-version': u'2013.1.4',
 u'openvswitch-version': u'ovs-vsctl (Open vSwitch) 1.9.3\nCompiled Oct 17 2013 10:38:04',
 u'uwm-url': u'http://10.0.0.33:8081',
 u'version': u'1.2',
 u'virl-version': u'0.10.3.1'}

STD server licensing:
{u'features': [u'Cariden.MATE.import', u'Cariden.MATE.export'],
 u'hostid': [u'HOSTNAME=virl', u'000c29467568'],
 u'product-capacity': 200,
 u'product-expires': None,
 u'product-license': u'CML_CORPORATE',
 u'product-usage': 0,
 u'uwm-url': u'http://10.0.0.33:8081/admin/licenses/'}

STD server autonetkit status:
{u'autonetkit-cisco-version': u'VIRL Configuration Engine EFT Release',
 u'autonetkit-version': u'autonetkit 0.8.42',
 u'version': u'1.0',
 u'virl-version': u'0.10.3.1'}

 

Cisco Employee

Hello Mohan,Thanks for the

Hello Mohan,

Thanks for the update.

 

 

4992
Views
0
Helpful
16
Replies
CreatePlease to create content