COT – Common OVF-Tool – Import of VCenter HTTPS Certificate

Background: Python-Requests wants to verify https-certificates, which makes sense to me.
– but my VCenter uses a self-signed Certificate

$ python
Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get("https://<vcenter-ip>")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 520, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 630, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 506, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='192.168.2.13', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),))

Don’t think about ignoring certificate errors 😉

Prerequisites

sudo apt-get install unzip

Download the certificate from the VCenter:

wget --no-check-certificate https://<vcenter-ip>/certs/download
mv download download.zip
unzip download.zip

Install the certificate:

cd certs
$ ls -l
total 8
-rw-r--r-- 1 administrator administrator 1294 Apr 29  2016 d2f59c0d.0
-rw-r--r-- 1 administrator administrator  702 Oct 15 14:36 d2f59c0d.r0

$ export REQUESTS_CA_BUNDLE=d2f59c0d.0

Now Python.requests validates the vCenter:

$ python
Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get("https://<vcenter-ip>")
<Response [200]>

Add the Export to ~/.bashrc

sudo vi ~/.bashrc

at the end of the file

export REQUESTS_CA_BUNDLE=~/certs/d2f59c0d.0

COT – Common OVF Tool

…this article should have been published at 2017-11-23, i forgot to press the right botton.

So, an update might be needed, anyone still interested in cot today?

Release 2.1 (2018-01-29) is the current release, containing some fixes (https://github.com/glennmatthews/cot/blob/master/CHANGELOG.rst)

  • COT installation using PIP

So, let’s go, following

administrator@lx-ubuntu:~$ sudo pip install cot
sudo: pip: command not found


thats embarassing.

Install PIP first

$ sudo apt-get update
$ sudo apt-get install python-pip

$ pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)

Try it again

$ sudo pip install cot
...
Successfully installed backports.shutil-get-terminal-size-1.0.0 colorlog-3.1.4 cot-2.1.0 ndg-httpsclient-0.5.1 pyvmomi-6.7.0.2018.9 verboselogs-1.7
$ cot --version
Common OVF Tool (COT), version 2.1.0
Copyright (C) 2013-2017 the COT project developers.
  • Optional: Argument-Completion

Not essential, but handy.

$sudo pip install argcomplete
...
Successfully installed argcomplete-1.9.4
  • COT helpers

A full installation requires so called „cot-helpers“ to use all features.

$ cot install-helpers --verify-only
Results:
-------------
COT manpages: DIRECTORY NOT FOUND: /usr/local/man/man1
fatdisk:      NOT FOUND
mkisofs:      NOT FOUND
ovftool:      NOT FOUND
qemu-img:     NOT FOUND
vmdktool:     NOT FOUND
$ sudo apt-get install xutils-dev
...
Setting up xutils-dev (1:7.7+5ubuntu1) ...
$ sudo cot install-helpers
...
Results:
-------------
COT manpages: already installed, no updates needed
fatdisk:      INSTALLATION FAILED: [Errno 1] Helper program './RUNME' exited with error 1
mkisofs:      version 2.1, present at /usr/bin/mkisofs
ovftool:      INSTALLATION FAILED: No support for automated installation of ovftool, as VMware requires a site login to
              download it. See https://www.vmware.com/support/developer/ovf/
qemu-img:     version 2.11.1, present at /usr/bin/qemu-img
vmdktool:     version 1.4, present at /usr/local/bin/vmdktool

[Errno 1] Unable to install some helpers

first fix „fatdisk“ which is used to inject configs into OVA-Files


$ sudo cp /usr/include/x86_64-linux-gnu/bits/stdint-intn.h /usr/include/x86_64-linux-gnu/bits/stdint-intn.h.bak
$ sudo vi /usr/include/x86_64-linux-gnu/bits/stdint-intn.h


uncomment

 typedef __int64_t int64_t;


to

//typedef __int64_t int64_t;


and run the helper-installation again:

$ sudo cot install-helpers
...
NOTICE  : Successfully installed 'fatdisk'
Results:
-------------
COT manpages: already installed, no updates needed
fatdisk:      successfully installed to /usr/local/bin/fatdisk, version 1.0
mkisofs:      version 2.1, present at /usr/bin/mkisofs
ovftool:      INSTALLATION FAILED: No support for automated installation of ovftool, as VMware requires a site login to
              download it. See https://www.vmware.com/support/developer/ovf/
qemu-img:     version 2.11.1, present at /usr/bin/qemu-img
vmdktool:     version 1.4, present at /usr/local/bin/vmdktool

[Errno 1] Unable to install some helpers


But it threw a lot of warnings..
warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 3 has type ‘int64_t {aka long long int}’ [-Wformat=]
… but it compiles… Memo: Try manual compilation…

$ mkdir fatdisk
$ cd fatdisk
$ wget -O fatdisk.tgz https://github.com/goblinhack/fatdisk/archive/master.tar.gz
$ tar zxf fatdisk.tgz
$ cd fatdisk-master/
$ ./RUNME

OFVTool of course has to be installed manually
Download from VMware: OVFTOOL-Download

Upload to the Linux-Server using Putty-SCP(PSCP) and public-key-authentication:

C:\Downloads&gt;pscp -scp -i C:\rh.priv.ppk VMware-ovftool-4.3.0-7948156-lin.x86_64.bundle administrator@192.168.56.101:
VMware-ovftool-4.3.0-7948 | 35623 kB | 35623.9 kB/s | ETA: 00:00:00 | 100%

At the Linux-Server

$ ls -l
total 35644
-rw-r--r-- 1 administrator administrator 36478864 Oct 15 18:36 VMware-ovftool-4.3.0-7948156-lin.x86_64.bundle
$ chmod a+x VMware-ovftool-4.3.0-7948156-lin.x86_64.bundle
$ sudo ./VMware-ovftool-4.3.0-7948156-lin.x86_64.bundle
...
Installing VMware OVF Tool component for Linux 4.3.0
    Configuring...
[######################################################################] 100%
Installation was successful.
administrator@lx-ubuntu:~$ cot install-helpers --verify-only
Results:
-------------
COT manpages: already installed, no updates needed
fatdisk:      version 1.0, present at /usr/local/bin/fatdisk
mkisofs:      version 2.1, present at /usr/bin/mkisofs
ovftool:      version 4.3, present at /usr/bin/ovftool
qemu-img:     version 2.11.1, present at /usr/bin/qemu-img
vmdktool:     version 1.4, present at /usr/local/bin/vmdktool

Linux: a start job is running for wait for Network to be configured

A new linux-VM takes ages to boot:

  • it waits for 2 minutes for the network to come up

Linux: Boot-Delay – network not up

  • Whats the reason for the delay?

„routable“ interfaces with not router-address provided by the dhcp-service never come up for the „networkd-wait-online“-service.

find more information at GitHub Issue 3752 – systemd-networkd can’t configure interface if dhcp server does not provide routers

administrator@lx-ubuntu:~$ networkctl
IDX LINK             TYPE               OPERATIONAL SETUP
  1 lo               loopback           carrier     unmanaged
  2 enp0s3           ether              routable    configuring
  3 enp0s8           ether              routable    configured

3 links listed.

State: forever „configuring“ not „configured“

administrator@lx-ubuntu:~$ networkctl status enp0s3
2: enp0s3
       Link File: /lib/systemd/network/99-default.link
    Network File: /run/systemd/network/10-netplan-enp0s3.network
            Type: ether
           State: routable (configuring)
            Path: pci-0000:00:03.0
          Driver: e1000
          Vendor: Intel Corporation
           Model: 82540EM Gigabit Ethernet Controller (PRO/1000 MT Desktop Adapter)
      HW Address: 08:00:27:37:db:ef (PCS Systemtechnik GmbH)
         Address: 192.168.56.101
                  fe80::a00:27ff:fe37:dbef
  • Background information

„enp0s3“ is a virtual-box „host-only“ network with no routing provided

VirtualBox – host-local network

  • Action Plan: „Set this interface to not be checked.“
  1. locate the systemd-unit-file:
  2. Path: /run/systemd/generator/network-online.target.wants
    File: systemd-networkd-wait-online.service

  3. Edit (sudo) file
  4. add „ignore“-Option
  5. docs: networkd wait online service

sudo vi /run/systemd/generator/network-online.target.wants/systemd-networkd-wait-online.service

The service specific configuration options are configured in the „[Service]“ section.

[Service]
Type=oneshot
ExecStart=/lib/systemd/systemd-networkd-wait-online --ignore=enp0s3
RemainAfterExit=yes
  • reboot

Now the VM boots like lightning.

Ubuntu – Install Ansible

It should be easy, and it is – but no single-command installation:

administrator@lx-ubuntu:$ sudo apt-get install ansible
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package ansible

Look at the Ubuntu Packet-Search for Ansible:

  • it’s in the „universe“-repository

Add the „universe“-repository:

administrator@lx-ubuntu:$ sudo add-apt-repository universe
'universe' distribution component enabled for all sources.
Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease
Get:2 http://archive.ubuntu.com/ubuntu bionic-security InRelease [83.2 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [8,570 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic/universe Translation-en [4,941 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [84.5 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic-security/universe Translation-en [46.4 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [556 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [143 kB]
Fetched 14.5 MB in 18s (824 kB/s)
Reading package lists... Done

Now it works:

administrator@lx-ubuntu:$ sudo apt-get install ansible
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  ieee-data libpython-stdlib libpython2.7-minimal libpython2.7-stdlib python python-asn1crypto python-certifi
  python-cffi-backend python-chardet python-crypto python-cryptography python-enum34 python-httplib2 python-idna
  python-ipaddress python-jinja2 python-jmespath python-kerberos python-libcloud python-lockfile python-markupsafe
  python-minimal python-netaddr python-openssl python-paramiko python-pkg-resources python-pyasn1 python-requests
  python-selinux python-simplejson python-six python-urllib3 python-xmltodict python-yaml python2.7 python2.7-minimal
Suggested packages:
  cowsay sshpass python-doc python-tk python-crypto-doc python-cryptography-doc python-cryptography-vectors
  python-enum34-doc python-jinja2-doc python-lockfile-doc ipython python-netaddr-docs python-openssl-doc
  python-openssl-dbg python-gssapi python-setuptools python-socks python-ntlm python2.7-doc binutils binfmt-support
Recommended packages:
  python-winrm
The following NEW packages will be installed:
  ansible ieee-data libpython-stdlib libpython2.7-minimal libpython2.7-stdlib python python-asn1crypto python-certifi
  python-cffi-backend python-chardet python-crypto python-cryptography python-enum34 python-httplib2 python-idna
  python-ipaddress python-jinja2 python-jmespath python-kerberos python-libcloud python-lockfile python-markupsafe
  python-minimal python-netaddr python-openssl python-paramiko python-pkg-resources python-pyasn1 python-requests
  python-selinux python-simplejson python-six python-urllib3 python-xmltodict python-yaml python2.7 python2.7-minimal
0 upgraded, 37 newly installed, 0 to remove and 3 not upgraded.
Need to get 12.1 MB of archives.
After this operation, 79.4 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 libpython2.7-minimal amd64 2.7.15~rc1-1 [334 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python2.7-minimal amd64 2.7.15~rc1-1 [1,292 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-minimal amd64 2.7.15~rc1-1 [28.1 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 libpython2.7-stdlib amd64 2.7.15~rc1-1 [1,910 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python2.7 amd64 2.7.15~rc1-1 [238 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 libpython-stdlib amd64 2.7.15~rc1-1 [7,620 B]
Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python amd64 2.7.15~rc1-1 [140 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-asn1crypto all 0.24.0-1 [72.7 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-cffi-backend amd64 1.11.5-1 [63.4 kB]
Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-enum34 all 1.1.6-2 [34.8 kB]
Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-idna all 2.6-1 [32.4 kB]
Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-ipaddress all 1.0.17-1 [18.2 kB]
Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-six all 1.11.0-2 [11.3 kB]
Get:14 http://archive.ubuntu.com/ubuntu bionic-security/main amd64 python-cryptography amd64 2.1.4-1ubuntu1.2 [221 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-markupsafe amd64 1.0-1build1 [13.0 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-jinja2 all 2.10-1 [94.6 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pyasn1 all 0.4.2-3 [46.7 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-paramiko all 2.0.0-1ubuntu1 [110 kB]
Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pkg-resources all 39.0.1-2 [128 kB]
Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-yaml amd64 3.12-1build2 [115 kB]
Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-crypto amd64 2.6.1-8ubuntu2 [244 kB]
Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-httplib2 all 0.9.2+dfsg-1 [34.6 kB]
Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 ieee-data all 20180204.1 [1,539 kB]
Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-netaddr all 0.7.19-1 [213 kB]
Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 ansible all 2.5.1+dfsg-1 [3,197 kB]
Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-certifi all 2018.1.18-2 [144 kB]
Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-chardet all 3.0.4-1 [80.3 kB]
Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-jmespath all 0.9.3-1ubuntu1 [21.2 kB]
Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-kerberos amd64 1.1.14-1 [22.5 kB]
Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-urllib3 all 1.22-1 [85.1 kB]
Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-requests all 2.18.4-2 [58.3 kB]
Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-lockfile all 1:0.12.2-2 [14.6 kB]
Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-simplejson amd64 3.13.2-1 [61.2 kB]
Get:34 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-libcloud all 2.2.1-1 [1,235 kB]
Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-openssl all 17.5.0-1ubuntu1 [41.3 kB]
Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-selinux amd64 2.7-2build2 [138 kB]
Get:37 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-xmltodict all 0.11.0-1 [10.3 kB]
Fetched 12.1 MB in 18s (672 kB/s)
Extracting templates from packages: 100%
Selecting previously unselected package libpython2.7-minimal:amd64.
(Reading database ... 102195 files and directories currently installed.)
Preparing to unpack .../0-libpython2.7-minimal_2.7.15~rc1-1_amd64.deb ...
Unpacking libpython2.7-minimal:amd64 (2.7.15~rc1-1) ...
Selecting previously unselected package python2.7-minimal.
Preparing to unpack .../1-python2.7-minimal_2.7.15~rc1-1_amd64.deb ...
Unpacking python2.7-minimal (2.7.15~rc1-1) ...
Selecting previously unselected package python-minimal.
Preparing to unpack .../2-python-minimal_2.7.15~rc1-1_amd64.deb ...
Unpacking python-minimal (2.7.15~rc1-1) ...
Selecting previously unselected package libpython2.7-stdlib:amd64.
Preparing to unpack .../3-libpython2.7-stdlib_2.7.15~rc1-1_amd64.deb ...
Unpacking libpython2.7-stdlib:amd64 (2.7.15~rc1-1) ...
Selecting previously unselected package python2.7.
Preparing to unpack .../4-python2.7_2.7.15~rc1-1_amd64.deb ...
Unpacking python2.7 (2.7.15~rc1-1) ...
Selecting previously unselected package libpython-stdlib:amd64.
Preparing to unpack .../5-libpython-stdlib_2.7.15~rc1-1_amd64.deb ...
Unpacking libpython-stdlib:amd64 (2.7.15~rc1-1) ...
Setting up libpython2.7-minimal:amd64 (2.7.15~rc1-1) ...
Setting up python2.7-minimal (2.7.15~rc1-1) ...
Linking and byte-compiling packages for runtime python2.7...
Setting up python-minimal (2.7.15~rc1-1) ...
Selecting previously unselected package python.
(Reading database ... 102942 files and directories currently installed.)
Preparing to unpack .../00-python_2.7.15~rc1-1_amd64.deb ...
Unpacking python (2.7.15~rc1-1) ...
Selecting previously unselected package python-asn1crypto.
Preparing to unpack .../01-python-asn1crypto_0.24.0-1_all.deb ...
Unpacking python-asn1crypto (0.24.0-1) ...
Selecting previously unselected package python-cffi-backend.
Preparing to unpack .../02-python-cffi-backend_1.11.5-1_amd64.deb ...
Unpacking python-cffi-backend (1.11.5-1) ...
Selecting previously unselected package python-enum34.
Preparing to unpack .../03-python-enum34_1.1.6-2_all.deb ...
Unpacking python-enum34 (1.1.6-2) ...
Selecting previously unselected package python-idna.
Preparing to unpack .../04-python-idna_2.6-1_all.deb ...
Unpacking python-idna (2.6-1) ...
Selecting previously unselected package python-ipaddress.
Preparing to unpack .../05-python-ipaddress_1.0.17-1_all.deb ...
Unpacking python-ipaddress (1.0.17-1) ...
Selecting previously unselected package python-six.
Preparing to unpack .../06-python-six_1.11.0-2_all.deb ...
Unpacking python-six (1.11.0-2) ...
Selecting previously unselected package python-cryptography.
Preparing to unpack .../07-python-cryptography_2.1.4-1ubuntu1.2_amd64.deb ...
Unpacking python-cryptography (2.1.4-1ubuntu1.2) ...
Selecting previously unselected package python-markupsafe.
Preparing to unpack .../08-python-markupsafe_1.0-1build1_amd64.deb ...
Unpacking python-markupsafe (1.0-1build1) ...
Selecting previously unselected package python-jinja2.
Preparing to unpack .../09-python-jinja2_2.10-1_all.deb ...
Unpacking python-jinja2 (2.10-1) ...
Selecting previously unselected package python-pyasn1.
Preparing to unpack .../10-python-pyasn1_0.4.2-3_all.deb ...
Unpacking python-pyasn1 (0.4.2-3) ...
Selecting previously unselected package python-paramiko.
Preparing to unpack .../11-python-paramiko_2.0.0-1ubuntu1_all.deb ...
Unpacking python-paramiko (2.0.0-1ubuntu1) ...
Selecting previously unselected package python-pkg-resources.
Preparing to unpack .../12-python-pkg-resources_39.0.1-2_all.deb ...
Unpacking python-pkg-resources (39.0.1-2) ...
Selecting previously unselected package python-yaml.
Preparing to unpack .../13-python-yaml_3.12-1build2_amd64.deb ...
Unpacking python-yaml (3.12-1build2) ...
Selecting previously unselected package python-crypto.
Preparing to unpack .../14-python-crypto_2.6.1-8ubuntu2_amd64.deb ...
Unpacking python-crypto (2.6.1-8ubuntu2) ...
Selecting previously unselected package python-httplib2.
Preparing to unpack .../15-python-httplib2_0.9.2+dfsg-1_all.deb ...
Unpacking python-httplib2 (0.9.2+dfsg-1) ...
Selecting previously unselected package ieee-data.
Preparing to unpack .../16-ieee-data_20180204.1_all.deb ...
Unpacking ieee-data (20180204.1) ...
Selecting previously unselected package python-netaddr.
Preparing to unpack .../17-python-netaddr_0.7.19-1_all.deb ...
Unpacking python-netaddr (0.7.19-1) ...
Selecting previously unselected package ansible.
Preparing to unpack .../18-ansible_2.5.1+dfsg-1_all.deb ...
Unpacking ansible (2.5.1+dfsg-1) ...
Selecting previously unselected package python-certifi.
Preparing to unpack .../19-python-certifi_2018.1.18-2_all.deb ...
Unpacking python-certifi (2018.1.18-2) ...
Selecting previously unselected package python-chardet.
Preparing to unpack .../20-python-chardet_3.0.4-1_all.deb ...
Unpacking python-chardet (3.0.4-1) ...
Selecting previously unselected package python-jmespath.
Preparing to unpack .../21-python-jmespath_0.9.3-1ubuntu1_all.deb ...
Unpacking python-jmespath (0.9.3-1ubuntu1) ...
Selecting previously unselected package python-kerberos.
Preparing to unpack .../22-python-kerberos_1.1.14-1_amd64.deb ...
Unpacking python-kerberos (1.1.14-1) ...
Selecting previously unselected package python-urllib3.
Preparing to unpack .../23-python-urllib3_1.22-1_all.deb ...
Unpacking python-urllib3 (1.22-1) ...
Selecting previously unselected package python-requests.
Preparing to unpack .../24-python-requests_2.18.4-2_all.deb ...
Unpacking python-requests (2.18.4-2) ...
Selecting previously unselected package python-lockfile.
Preparing to unpack .../25-python-lockfile_1%3a0.12.2-2_all.deb ...
Unpacking python-lockfile (1:0.12.2-2) ...
Selecting previously unselected package python-simplejson.
Preparing to unpack .../26-python-simplejson_3.13.2-1_amd64.deb ...
Unpacking python-simplejson (3.13.2-1) ...
Selecting previously unselected package python-libcloud.
Preparing to unpack .../27-python-libcloud_2.2.1-1_all.deb ...
Unpacking python-libcloud (2.2.1-1) ...
Selecting previously unselected package python-openssl.
Preparing to unpack .../28-python-openssl_17.5.0-1ubuntu1_all.deb ...
Unpacking python-openssl (17.5.0-1ubuntu1) ...
Selecting previously unselected package python-selinux.
Preparing to unpack .../29-python-selinux_2.7-2build2_amd64.deb ...
Unpacking python-selinux (2.7-2build2) ...
Selecting previously unselected package python-xmltodict.
Preparing to unpack .../30-python-xmltodict_0.11.0-1_all.deb ...
Unpacking python-xmltodict (0.11.0-1) ...
Setting up ieee-data (20180204.1) ...
Processing triggers for mime-support (3.60ubuntu1) ...
Processing triggers for man-db (2.8.3-2) ...
Setting up libpython2.7-stdlib:amd64 (2.7.15~rc1-1) ...
Setting up python2.7 (2.7.15~rc1-1) ...
Setting up libpython-stdlib:amd64 (2.7.15~rc1-1) ...
Setting up python (2.7.15~rc1-1) ...
Setting up python-idna (2.6-1) ...
Setting up python-simplejson (3.13.2-1) ...
Setting up python-yaml (3.12-1build2) ...
Setting up python-asn1crypto (0.24.0-1) ...
Setting up python-crypto (2.6.1-8ubuntu2) ...
Setting up python-pyasn1 (0.4.2-3) ...
Setting up python-netaddr (0.7.19-1) ...
Setting up python-xmltodict (0.11.0-1) ...
Setting up python-jmespath (0.9.3-1ubuntu1) ...
Setting up python-certifi (2018.1.18-2) ...
Setting up python-kerberos (1.1.14-1) ...
Setting up python-pkg-resources (39.0.1-2) ...
Setting up python-markupsafe (1.0-1build1) ...
Setting up python-httplib2 (0.9.2+dfsg-1) ...
Setting up python-cffi-backend (1.11.5-1) ...
Setting up python-six (1.11.0-2) ...
Setting up python-selinux (2.7-2build2) ...
Setting up python-enum34 (1.1.6-2) ...
Setting up python-lockfile (1:0.12.2-2) ...
Setting up python-ipaddress (1.0.17-1) ...
Setting up python-urllib3 (1.22-1) ...
Setting up python-chardet (3.0.4-1) ...
Setting up python-jinja2 (2.10-1) ...
Setting up python-cryptography (2.1.4-1ubuntu1.2) ...
Setting up python-requests (2.18.4-2) ...
Setting up python-openssl (17.5.0-1ubuntu1) ...
Setting up python-paramiko (2.0.0-1ubuntu1) ...
Setting up ansible (2.5.1+dfsg-1) ...
Setting up python-libcloud (2.2.1-1) ...

Which version?

  • as „packages.ubuntu.com“ said: 2.5.1
administrator@lx-ubuntu:$ ansible --version
ansible 2.5.1
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/administrator/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]

Linux – NetPlan Interface Configuration

Just added a second NIC to a Linux-VM…

  • new config-style – new luck 🙂
administrator@lx-ubuntu:/etc/netplan$ cat /etc/network/interfaces
# ifupdown has been replaced by netplan(5) on this system. See
# /etc/netplan for current configuration.
# To re-enable ifupdown on this system, you can run:
#    sudo apt install ifupdown

Ok, it seems to be easy to get back to good-old network-config-style.

But after reading https://netplan.io/examples this seems to be very handy.

The initial „netplan-file“ references only the initial NIC:

administrator@lx-ubuntu:~$ cd /etc/netplan/
administrator@lx-ubuntu:/etc/netplan$ cat 50-cloud-init.yaml
# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    ethernets:
        enp0s3:
            addresses: []
            dhcp4: true
    version: 2

What’s the ID of the new NIC?

  • it’s „enp0s8“
administrator@lx-ubuntu:/etc/netplan$ ifconfig
enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.56.104  netmask 255.255.255.0  broadcast 192.168.56.255
        inet6 fe80::a00:27ff:fe37:dbef  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:37:db:ef  txqueuelen 1000  (Ethernet)
        RX packets 989  bytes 119568 (119.5 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 704  bytes 105969 (105.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.3.15  netmask 255.255.255.0  broadcast 10.0.3.255
        inet6 fe80::a00:27ff:fe8d:ab00  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:8d:ab:00  txqueuelen 1000  (Ethernet)
        RX packets 2  bytes 1180 (1.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5  bytes 944 (944.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 2032  bytes 123200 (123.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2032  bytes 123200 (123.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Create a backup

administrator@lx-ubuntu:/etc/netplan$ sudo cp 50-cloud-init.yaml 50-cloud-init.yaml.bak

Modify the Netplan-File:

administrator@lx-ubuntu:/etc/netplan$ sudo vi 50-cloud-init.yaml

administrator@lx-ubuntu:/etc/netplan$ cat 50-cloud-init.yaml
# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    ethernets:
        enp0s3:
            addresses: []
            dhcp4: true
        enp0s8:
            addresses: []
            dhcp4: true
    version: 2

Activate the new Config:

administrator@lx-ubuntu:/etc/netplan$ sudo netplan apply

Verify the result

administrator@lx-ubuntu:/etc/netplan$ ifconfig
enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.56.104  netmask 255.255.255.0  broadcast 192.168.56.255
        inet6 fe80::a00:27ff:fe37:dbef  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:37:db:ef  txqueuelen 1000  (Ethernet)
        RX packets 989  bytes 119568 (119.5 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 704  bytes 105969 (105.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.3.15  netmask 255.255.255.0  broadcast 10.0.3.255
        inet6 fe80::a00:27ff:fe8d:ab00  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:8d:ab:00  txqueuelen 1000  (Ethernet)
        RX packets 2  bytes 1180 (1.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5  bytes 944 (944.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 2032  bytes 123200 (123.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2032  bytes 123200 (123.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Amazon AWS – Use IAM-role to authenticate python-script on EC2-Instance to S3-API

Today i wanted to stop using hard-coded S3-Credentials in my scripts running on EC2-Instances.

Create an IAM-Role

  1. Services
  2. IAM
  3. Roles
  4. [Create Role]
  • Select type of trusted entity
    • AWS-Service
    • EC2
    • [Next: Permissions]
  • Attach permissions policies
    • AmazonS3ReadOnlyAccess
  • Review
    • Role name: READ_S3

IAM-role: AWS – EC2

role-policy: s3 – ead-only

role: review & set name

Attach Role to EC2-Instance

  1. Services
  2. EC2
  3. Instances
  4. Instance Settings
  • attach IAM-Role
    • READ_S3

EC2-instance: attach IAM Role

Instance: attach Role „READ_S3“

Prepare Linux Setup
Within the EC2-Instance to which the IAM-Role has been attached.

ubuntu@ip-10-2-0-193:~$ sudo apt-get install -y python-pip
ubuntu@ip-10-2-0-193:~$ sudo apt install -y awscli
ubuntu@ip-10-2-0-193:~$ pip install boto3
ubuntu@ip-10-2-0-193:~$ pip install requests

Check Access to „meta-data“ => „IAM“ => „Security-Credentials“
The Script has to know the name of the role to use.

ubuntu@ip-10-2-0-193:~$ curl http://169.254.169.254/latest/meta-data/iam/security-credentials/READ_S3
{
  "Code" : "Success",
  "LastUpdated" : "2018-02-27T20:16:38Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ASIAI5EHDXGCAQBR7AA",
  "SecretAccessKey" : "6qWNxrTDU2FTynEkkJHl4pADQ4/xdQqgr89tF8x",
  "Token" : "FQoDYXdzEMX//////////wEaDBVs9pF5ec5XdDPiFCK3A5zquC32puTTqR9jV2BiViEBtagFXJ13++wq+lSUoCxL7sZwSCwa9njCWppjv5ShoKdqfkAWWivecZ18Px8SLoO7T83pZCeyri+WKKVdzNrt5tvrftxxlRVMvj+/Hy683KJi4GeheOEAC0XuiNwFC0IMM5xwJatdzXlann6I/A1zLSC5p/iydHPug9YvgEn3+cwTa2CCBNjzk4Sh0NsGkdhVggjZNcYe/jWSxdcCJJNc5cSTz76z2G+vFppbWnL0Rz4iKGRVXXs81upazvtbheffVWApUiL0ULhS2jrxEzFioPNX73AGlwaNM9jD0Zk5N2BSQuQE9yxyKcetkcCZQ85B4pkmxgldPTStQng+zeYv/rAZjyURFHGHTPhOLaHnnZyn89EfjThZ8rYhq1RJpu8Gqx9Z58Jclyj0YiHQe6IfF77N8PBLc0lGu7iT3yc6cp3n2kdsmuK3N3+WmRQI0G+OMJvMSx6eH31IVgwF+nEwkl0epW13D+3RQEdaCwm9b5wV9UFBkXtI4XPgF08gS2qwiyRO25ztuWD1USq9ZchrmJIOMHO4BX1XpOmf0M9drpbmIYbP4EoovnW1AU=",
  "Expiration" : "2018-02-28T02:48:10Z"

Fetch the IAM-Role-Credentials using python.requests()
Attention: recreated the Role in the meantime, so the credentials have been modified.

ubuntu@ip-10-2-0-193:~$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> import requests
>>> import boto3

>>> r = requests.get('http://169.254.169.254/latest/meta-data/iam/security-credentials/READ_S3')
>>> cred = r.json()
>>> print cred
{u'Code': u'Success', u'LastUpdated': u'2018-02-27T20:33:14Z', u'AccessKeyId': u'ASIAIRKJFTEBQLSFR6A', u'SecretAccessKey': u'USXXJb1zoCa13FBRrUR5AWwaH0ymo+af8DmW7eQ', u'Token': u'FQoDYXdzEMb//////////wEaDGxRlk0fbmRHnfL8mSK3A0tyJcQ2JgXY+fEnjFioduAnU/FRhth+DhLHzJWnwSw0/isO/tP72wFkneklld1JtAkRkRxH9SjRwqWKpybtW6cEITEksP976Ym7sjxoT+NglC4KNk9DfaRgvJgB8EqDqgG1oZHoJ91O+US+vxPlLAifh7Fvepuw+2nmwPDpOIPz7H6Q2pAr/i7AEncyMGCwZejVhnY5lk8+aDmYMqS7ymfe1n/DyjwEfiYH4kFJplOtubXMsc+rlTg2KtMSovMRX4h+SrdH6bpVpT4DeIvivyH5ABFL66YMyW76P8GGdaGtb8ohxQZ3zeR1gGYs5WEBcziZVEGJDEopKLSUgqgkC1dUFLk0XLfB8vaEGjNsaaT3gaRm2AWAEVonLFyUr+uWvdxptOmFPIMUwUlNitXJkqG44fQEuOw5oDHANJ7zZtPYrWQUhmJzHubJZbvJ313GlPfUQzs7bkDBxoBXIgKIPMpK09uxh48iEByLdqoHyAtkL4qWddQVywuEgEH7Qw/X1/20ngaXqVVSjEa3hu/98EvxKOoushMpeLiG1JyxZJR5nQGH/YgxC17t2ipzsrWPepPkZfz5aAo3IDX1AU=', u'Expiration': u'2018-02-28T02:50:12Z', u'Type': u'AWS-HMAC'}
>>>

Use the credentials to create a Session-Object

>>> session = boto3.Session(
...     aws_access_key_id=cred["AccessKeyId"],
...     aws_secret_access_key=cred["SecretAccessKey"],
...     aws_session_token=cred["Token"],
... )
>>>

Read all S3-Buckets, print out their attributes („Name“)

>>> s3 = session.resource('s3', region_name='us-east-1')
>>>
>>> for b in s3.buckets.all():
...   print b
...
s3.Bucket(name='allones')
>>>

Is this all?

  • Only one Bucket?

Yes – and it’s located in Frankfurt.

S3: Buckets

Further reading:
Python API for Amazon S3: Examples

Cisco IOS Service-Containers: Run an x86-VM inside a Router (Part 5: Install the Service-VM)

The CSR1000V-Router has already been prepared (Create a CSR1000V-Instance with nested Virtualiation support) and in (Part 4: Package the Service-VM into an OVA) we copied the OVA-Image of our „ubuntu-server“-VM to this Router.

Add the internal Network-Interface between Router and Service-VM

SERVICECONTAINER#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
SERVICECONTAINER(config)#int virtualportgroup 0
*Feb  3 19:05:17.206: %LINEPROTO-5-UPDOWN: Line protocol on Interface VirtualPortGroup0, changed state to up
SERVICECONTAINER(config-if)#ip addr 192.168.0.1 255.255.255.0
SERVICECONTAINER(config-if)#exit

SERVICECONTAINER(config)#virtual-service
SERVICECONTAINER(config-virt-serv-global)#signing level unsigned
% Support for unsigned packages has been user-enabled. Unsigned packages are not endorsed by Cisco Systems, Inc. User assumes all responsibility
SERVICECONTAINER(config-virt-serv-global)#ex

SERVICECONTAINER(config)#ip dhcp pool SERVICE
SERVICECONTAINER(dhcp-config)# network 192.168.0.0 255.255.255.0
SERVICECONTAINER(dhcp-config)# exit

SERVICECONTAINER(config)#virtual-service UBUNTU
SERVICECONTAINER(config-virt-serv)#vnic gateway virtualPortGroup 0
SERVICECONTAINER(config-virt-serv-vnic)#guest ip address 192.168.0.2
SERVICECONTAINER(config-virt-serv-vnic)#end
SERVICECONTAINER#
*Feb  3 19:06:27.474: %SYS-5-CONFIG_I: Configured from console by console

Install the Service-VM

SERVICECONTAINER#debug virtual-service all
virtual service all debugging is on
SERVICECONTAINER#term mon
% Console already monitors
SERVICECONTAINER#term width 0
SERVICECONTAINER#virtual-service install name UBUNTU package bootflash:ubuntu1604.ova
Installing package 'bootflash:/ubuntu1604.ova' for virtual-service 'UBUNTU'. Once the install has finished, the VM may be activated. Use 'show virtual-service list' for progress.

*Feb  3 19:07:43.518: VIRTUAL-SERVICE [UBUNTU]: Sending install req for [UBUNTU], path=bootflash:/ubuntu1604.ova, uri= uid=0
*Feb  3 19:07:43.529: VIRTUAL-INSTANCE: Message sent for INSTALL TDL request: Virtual-instance name: UBUNTU, UID: 0
*Feb  3 19:07:43.529: VIRTUAL-SERVICE: Started response timer for tid DD000001 - 30 minutes
*Feb  3 19:05:14.205: %IOSXE-4-PLATFORM: R0/0: kernel: dev->name [intsvc0]: dev_entry not populated

SERVICECONTAINER#show virtual-service list
System busy installing virtual-service 'UBUNTU'. The request may take several minutes...
Virtual Service List:


Name                    Status             Package Name
------------------------------------------------------------------------------
UBUNTU                  Installing         ubuntu1604.ova


*Feb  3 19:08:00.645: %VMAN-5-PACKAGE_SIGNING_LEVEL_ON_INSTALL: R0/0: vman: Package 'ubuntu1604.ova' for service container 'UBUNTU' is 'unsigned', signing level cached on original install is 'unsigned'
*Feb  3 19:08:09.216: VIRTUAL-SERVICE: Install response handler: VM[UBUNTU]: Owner IOSd trans_id 3707764737
*Feb  3 19:08:09.216: VIRTUAL-SERVICE [UBUNTU]: vm[UBUNTU] set owner [IOSd]
*Feb  3 19:08:09.225: VIRTUAL-SERVICE [UBUNTU]: application_name: 'ubuntu' application_vendor: '' application_version: '1.1'
*Feb  3 19:08:09.226: VIRTUAL-SERVICE [UBUNTU]: Default profile info: license_name: '', license_ver: ''
*Feb  3 19:08:09.226: VIRTUAL-SERVICE: Install pkg response for tid DD000001: rc=0, descr=Install Success
*Feb  3 19:08:09.226: VIRTUAL-SERVICE [UBUNTU]: License type: none, no license needed
*Feb  3 19:08:09.226: %VIRT_SERVICE-5-INSTALL_STATE: Successfully installed virtual service UBUNTU
*Feb  3 19:08:09.243: VIRTUAL-SERVICE: Received local transport activation request
*Feb  3 19:08:09.244: VIRTUAL-SERVICE: Enabling vman local transport

SERVICECONTAINER#show virtual-service list
Virtual Service List:

Name                    Status             Package Name
------------------------------------------------------------------------------
UBUNTU                  Installed          ubuntu1604.ova

*Feb  3 19:08:32.758: %ONEP_BASE-6-SS_ENABLED: ONEP: Service set Base was enabled by Default
*Feb  3 19:08:33.259: VIRTUAL-SERVICE: Local transport 'activation' request processed

Activate the installed Service-VM

SERVICECONTAINER#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
SERVICECONTAINER(config)#virtual-service UBUNTU
SERVICECONTAINER(config-virt-serv)#activate
SERVICECONTAINER(config-virt-serv)#end

% Activating virtual-service 'UBUNTU', this might take a few minutes. Use 'show virtual-service list' for progress.

*Feb  3 19:09:57.772: VIRTUAL-SERVICE [UBUNTU]: Activate CLI: appl->owner [IOSd]
*Feb  3 19:09:57.775: VIRTUAL-INSTANCE: Message sent for ACTIVATE TDL request: Virtual service name: UBUNTU, UID: 0
*Feb  3 19:09:57.775: VIRTUAL-SERVICE [UBUNTU]: Started virtual service (0) activate response timer - 30 minutes
*Feb  3 19:10:06.052: %SYS-5-CONFIG_I: Configured from console by console
*Feb  3 19:10:06.383: VIRTUAL-SERVICE [UBUNTU]: Activate response handler: got owner [IOSd]
*Feb  3 19:10:06.383: VIRTUAL-SERVICE: clnt_type 0: Interface counter is '1'
*Feb  3 19:10:06.383: VIRTUAL-SERVICE: Information for virtual port grp '0' is received
*Feb  3 19:10:06.384: VIRTUAL-SERVICE [UBUNTU]: Deliver intf response, vm =UBUNTU, counter=1
*Feb  3 19:10:06.384: VIRTUAL-SERVICE [UBUNTU]: Received interface id=0, type=1, state=1
*Feb  3 19:10:06.384: VIRTUAL-SERVICE [UBUNTU]: Received virtual port group interface 0 with service MAC 001e.e5b1.cfba, state: up
*Feb  3 19:10:06.385: VIRTUAL-INSTANCE: Message sent for IF MTU TDL message: appliance 'UBUNTU'
*Feb  3 19:10:06.385: VIRTUAL-SERVICE [UBUNTU]: Activate response handler: rsp_rc 0
*Feb  3 19:10:06.385: VIRTUAL-SERVICE [UBUNTU]: Deliver response: appliance_state 3 rsp_rc 0 if_notify name UBUNTU clnt_type 0 act_state 0
*Feb  3 19:10:06.385: %VIRT_SERVICE-5-ACTIVATION_STATE: Successfully activated virtual service UBUNTUconf t
*Feb  3 19:10:06.385: VIRTUAL-SERVICE [UBUNTU]: Stopped virtual service (1) response timer
*Feb  3 19:10:06.385: VIRTUAL-SERVICE: Delivered Virt-manager response message to virtual service 'UBUNTU' - Response: 'OK'
*Feb  3 19:10:06.385: VIRTUAL-SERVICE [UBUNTU]: set owner to 'IOSd' in appliance

SERVICECONTAINER#show virtual-service list
Virtual Service List:

Name                    Status             Package Name
------------------------------------------------------------------------------
UBUNTU                  Activated          ubuntu1604.ova

SERVICECONTAINER#show ip dhcp bind
Bindings from all pools not associated with VRF:
IP address      Client-ID/              Lease expiration        Type       State      Interface
                Hardware address/
                User name
192.168.0.2     001e.e5b1.cfba          Feb 04 2018 07:20 PM    Automatic  Active     VirtualPortGroup0

Access the VM using the (virtual) Serial-Console

SERVICECONTAINER#virtual-service connect name UBUNTU console
Connected to appliance. Exit using ^c^c^c

Ubuntu 16.04.3 LTS ubuntu-server ttyS0

ubuntu-server login: user
Password:
Last login: Sat Feb  3 20:23:27 CET 2018 on ttyS0
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-87-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

9 packages can be updated.
7 updates are security updates.


user@ubuntu-server:~$ who
user     ttyS0        2018-02-03 23:12

Logout: 3x [CTRL]+

user@ubuntu-server:~$ ^C
user@ubuntu-server:~$ ^C
user@ubuntu-server:~$ ^C

Access the VM using SSH via the internal Network

SERVICECONTAINER#ssh -l user 192.168.0.2
Password:
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-87-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

9 packages can be updated.
7 updates are security updates.

Last login: Sat Feb  3 23:12:21 2018
user@ubuntu-server:~$ who
user     pts/0        2018-02-03 23:13 (192.168.0.1)

Check the local python/NAPALM-Setup to get facts about the containing router

user@ubuntu-server:~$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> import napalm

>>> iosdriver = napalm.get_network_driver('ios')
dpass', optional_args={'port': 22, 'dest_file_system': 'bootflash:'})sword='rmon

>>> router.open()

>>> print router.get_facts()
{u'os_version': u'Virtual XE Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 16.7.1, RELEASE SOFTWARE (fc6)', u'uptime': 12840, u'interface_list': [u'GigabitEthernet1', u'GigabitEthernet2', u'GigabitEthernet3', u'VirtualPortGroup0'], u'vendor': u'Cisco', u'serial_number': u'9SNHUBYAML', u'model': u'CSR1000V', u'hostname': u'SERVICECONTAINER', u'fqdn': u'SERVICECONTAINER.lab.local'}

Cisco IOS Service-Containers: Run an x86-VM inside a Router (Part 4: Package the Service-VM into an OVA)

I’ll start at the KVM-Virtualization Host („Ubuntu Desktop“) again.

Folder Structure
I’ll create a folder „isr-service-container“ for common stuff (package.yaml, create_ova.sh) and a for each VM a dedicated sub-folder, today: „ubuntu-server16.04“

user@KVM-1:~$ mkdir isr-service-container
user@KVM-1:~$ cd isr-service-container
user@KVM-1:~/isr-service-container$ mkdir ubuntu-server16.04

Download the Cisco-provided „templates.tar“ from GitHub (GitHub: Templates.tar).
It contains:

  • package.yaml
  • create_ova.sh

Prepare a compressed virtual harddisk of the „Ubuntu-Server“-VM
Locate the original virtual harddisk:

user@KVM-1:~$ sudo ls /var/lib/libvirt/images/ -l
total 3238172
-rw------- 1 root root 21478375424 Feb  2 23:14 ubuntu-server16.04.qcow2

Convert the original „.qcow2“-File into a compressed „copy“:

user@KVM-1:~/isr-service-container$ sudo qemu-img convert -p -c -o compat=0.10 -O qcow2 /var/lib/libvirt/images/ubuntu-server16.04.qcow2 ./ubuntu-server16.04/ubuntu-server16.04.qcow2
    (100.00/100%)

user@KVM-1:~/isr-service-container$ cd ubuntu-server16.04/
user@KVM-1:~/isr-service-container/ubuntu-server16.04$ ls -lh
total 1,2G
-rw-r--r-- 1 root root 1,2G Feb  2 23:26 ubuntu-server16.04.qcow2

Change the Owner of this new file:

user@KVM-1:~/isr-service-container/ubuntu-server16.04$ sudo chown user:user ubuntu-server16.04.qcow2 
user@KVM-1:~/isr-service-container/ubuntu-server16.04$ ls -lh
total 1,2G
-rw-r--r-- 1 user user 1,2G Feb  2 23:26 ubuntu-server16.04.qcow2
user@KVM-1:~/isr-service-container/ubuntu-server16.04$ copy ../package.yaml ./package.yaml

I’ll change:

  • Description: „KVM Ubuntu 16.04 LTS“
  • resources/vcpu: 1
  • disk/file: ubuntu-server16.04.qcow2

The vCPU# got decreased since the CSR1000v only supports Service-VMs with one vCPU.

user@KVM-1:~/isr-service-container/ubuntu-server16.04$ joe ./package.yaml
manifest-version: 1.0

info:
  name: ubuntu
  description: "KVM Ubuntu 16.04 LTS"
  version: 1.1

app:
  # Indicate app type (vm, paas, lxc etc.,)
  apptype: vm

  resources:
   cpu: 10
   memory: 854016
   vcpu: 1

   disk:
    - target-dev: hdc
      file: ubuntu-server16.04.qcow2

   interfaces:
    - target-dev: net1

   serial:
    - console
    - aux

  # Specify runtime and startup
  startup:
    runtime: kvm
    boot-dev: hd

Another File „version.ver“ has to be created:

  • the „version“ must match the „manifest-version“ of the yaml-file.
user@KVM-1:~/isr-service-container/ubuntu-server16.04$ echo 1.0 > version.ver

Check the content of the VM-folder

user@KVM-1:~/isr-service-container/ubuntu-server16.04$ ls -l
total 1206788
-rw-rw-r-- 1 user user        437 Feb  3 20:13 package.yaml
-rw-r--r-- 1 user user 1239148032 Feb  2 23:26 ubuntu-server16.04.qcow2
-rw-rw-r-- 1 user user          4 Feb  2 23:33 version.ver

Create the OVA-File

user@KVM-1:~/isr-service-container/ubuntu-server16.04$ cd ..

user@KVM-1:~/isr-service-container$ ./create_ova.sh -mts 200000 -mfs 100000 ubuntu-server16.04
create_ova.sh v1.0(Linux) - Create a virtual-service OVA package

User inputs:
  Compress=(files > '100000M' if total 
            file size > '200000M')
  Directory=ubuntu-server16.04

Package name :  ubuntu
 Generating SHA1 on files...
Running SHA1 over all files in '/home/user/isr-service-container/ubuntu-server16.04' and
    creating manifest file ' ubuntu.mf', please wait...

Done creating ' ubuntu.mf' file
 ...Done Generating SHA1 on files
Creating ' ubuntu.ova' please wait...
package.yaml
ubuntu.mf
ubuntu-server16.04.qcow2
version.ver

'/home/user/isr-service-container/ubuntu-server16.04/ ubuntu.ova' created

Manifest Contents:
SHA1(package.yaml)= fb47cf5b764a7bb062561a4f67d830003f8e4d5a  
SHA1(ubuntu-server16.04.qcow2)= 732c8ac9dc81ab6b2695fe6c045bec4493d77168  
SHA1(version.ver)= 61652cd1568dcf2614df833eba241755eee34e89  

Copy the OVA-File to the Router:

user@KVM-1:~/isr-service-container$ scp ./ubuntu-server16.04/ubuntu.ova <ios-user>@<csr1000v-management-router-ip>:bootflash:ubuntu1604.ova
Password: <ios-password>
ubuntu.ova                                                                                         100% 1182MB   1.2MB/s   17:05    
Connection to 192.168.2.189 closed by remote host.

Check the ISR-Router:

SERVICECONTAINER#dir *.ova

Directory of bootflash:/
   22  -rw-  1239152640   Feb 3 2018 19:00:42 +00:00  ubuntu1604.ova
7897796608 bytes total (4507389952 bytes free)

Cisco IOS Service-Containers: Run an x86-VM inside a Router (Part 3: Create the Service-VM)

Create the „Ubuntu Server“-VM using KVM on top of the „Ubuntu Desktop“-VM
Run the „Virtual Machine Manager“

Virtual Machine Manager

Create a new virtual Machine:
Virtual Machine Manager – create a new VM

(1) Create a new virtual Machine

vmm – 1 – Create a new virtual machine

(2) Select the Installation ISO
* and deselect „auto OS Detection“

vmm – 2 – Use ISO Image

(3) Choose Memory and CPU Settings

vmm – 3 – Choose Memory and CPU settings

(4) Create a virtual Hard-Disk for the VM

vmm – 4 – Enable storage for this VM

(5) Give a name to the VM

vmm – 5 – Ready to begin the installation

Boot the KVM-VM („Ubuntu Server“)

0 – install Ubuntu Server

Basic Setup:

  • Language: Englisch
  • my territory: other/Europe/Germany
  • locale: en_US.UTF_8
  • Keyboard: German
  • hostname: ubuntu-server
  • user / password
  • no home-directory encryption
  • timezone Europe/Berlin
  • use entire virtual disk (no need for „LVM…“ i think)
  • no HTTP-Proxy
  • no automatic updates
  • [x] SSH-Server
  • install GRUB

After the first login, as always:
sudo apt-get update && sudo apt-get upgrade –y
sudo reboot

Enable the Virtual-„Console“-Port of the Ubuntu-Server-VM

  • enable a virtual „CON“ serial-port at the linux vm

sudo joe /lib/systemd/system/ttyS0.service

Example-Script for the 16.04 System-CTL Service:

[Unit]
Description=Serial Console Service

[Service]
ExecStart=/sbin/getty -L 9600 ttyS0 vt102
Restart=always

[Install]
WantedBy=multi-user.target

Reload systemctl, enable the TTY-service and then start it:

sudo systemctl daemon-reload
sudo systemctl enable ttyS0
sudo systemctl start ttyS0

Verify the service state:
user@ubuntu-server:~$ sudo systemctl status ttyS0
● ttyS0.service - Serial Console Service
Loaded: loaded (/lib/systemd/system/ttyS0.service; enabled; vendor preset: en
Active: active (running) since Sat 2018-02-03 20:21:20 CET; 1h 19min ago
Main PID: 831 (login)
Tasks: 0
Memory: 672.0K
CPU: 196ms
CGroup: /system.slice/ttyS0.service
‣ 831 /bin/login --

Add your „Network Application“ to the VM:
I want to use my Service-Container for network-management (Centralized access to device-configuration and other state-information using NAPALM and Linux: SNMP with Python ):
sudo apt-get install -y joe
sudo apt-get install –y libsnmp-dev snmp-mibs-downloader
sudo apt-get install –y gcc python-dev python-pip libssl-dev libffi-dev python-cffi
pip install easysnmp
pip install napalm

That’s all
Shutdown the VM