Cloud Computing at IFCA

1. Introduction

This is a beta service, since the deployment and development is ongoing. However, access is granted to certain users to test the functionality.

Check this document for updates frequently, since changes in the service may occur.

IFCA uses OpenStack (Havana version) for managing the cloud service, which provides several ways to access:

• Web dashboard

• OpenStack API, that can be used with the nova command line interface.

• OCCI API, that can be used with rOCCI client

This documents focuses on the usage of the nova command. This and the web dashboard are the recommended client tools to use with our infrastructure.

For the latest up-to-date documentation, check also the Openstack official documentation

2. Access with the nova commands

2.1. Installing the client

If you are on debian/ubuntu, you can install the nova client with apt-get:

$ sudo apt-get install python-novaclient

RedHat based distributions also include the package in EPEL repositories. You can install it with yum:

$ yum install python-novaclient

Alternatively, you can use pip to install the python package directly:

$ pip install python-novaclient

The GRIDUI Cluster (SL6 machines) include a ready to use installation for IFCA local users.

2.2. Getting the credentials

The easiest way to obtain your credentials is using the dashboard of OpenStack: login into Web portal and go to your Access & Security area (the link is in the section Projects) and then to the API Access subsection. There you will a button to download the OpenStack RC File that can be used with the nova commands.

Download or copy the configuration file for OpenStack to your directory. Source it in order to have the environment ready, providing the corresponding password:

$ . openrc.sh
Please enter your OpenStack Password:

Now you should be able to execute the commands to access the CLOUD resources.

Try that your environment is correct with the nova endpoints command. It should return all the services available in the OpenStack installation:

$ nova endpoints
+-------------+---------------------------------------------------------------+
| nova-volume | Value                                                         |
+-------------+---------------------------------------------------------------+
| adminURL    | http://cloud.ifca.es:8776/v1/c725b18d8d0643e7b410dc5a8d9ab554 |
| internalURL | http://cloud.ifca.es:8776/v1/c725b18d8d0643e7b410dc5a8d9ab554 |
| publicURL   | http://cloud.ifca.es:8776/v1/c725b18d8d0643e7b410dc5a8d9ab554 |
| region      | RegionOne                                                     |
+-------------+---------------------------------------------------------------+
+-------------+-------------------------------+
| glance      | Value                         |
+-------------+-------------------------------+
| adminURL    | http://glance.ifca.es:9292/v1 |
| internalURL | http://glance.ifca.es:9292/v1 |
| publicURL   | http://glance.ifca.es:9292/v1 |
| region      | RegionOne                     |
+-------------+-------------------------------+
+-------------+-----------------------------------------------------------------+
| nova        | Value                                                           |
+-------------+-----------------------------------------------------------------+
| adminURL    | http://cloud.ifca.es:8774/v1.1/c725b18d8d0643e7b410dc5a8d9ab554 |
| internalURL | http://cloud.ifca.es:8774/v1.1/c725b18d8d0643e7b410dc5a8d9ab554 |
| publicURL   | http://cloud.ifca.es:8774/v1.1/c725b18d8d0643e7b410dc5a8d9ab554 |
| region      | RegionOne                                                       |
| serviceName | nova                                                            |
+-------------+-----------------------------------------------------------------+
+-------------+------------------------------------------+
| ec2         | Value                                    |
+-------------+------------------------------------------+
| adminURL    | http://cloud.ifca.es:8773/services/Admin |
| internalURL | http://cloud.ifca.es:8773/services/Cloud |
| publicURL   | http://cloud.ifca.es:8773/services/Cloud |
| region      | RegionOne                                |
+-------------+------------------------------------------+
+-------------+------------------------------------+
| keystone    | Value                              |
+-------------+------------------------------------+
| adminURL    | http://keystone.ifca.es:35357/v2.0 |
| internalURL | http://keystone.ifca.es:5000/v2.0  |
| publicURL   | http://keystone.ifca.es:5000/v2.0  |
| region      | RegionOne                          |
+-------------+------------------------------------+

3. Managing machines

The cloud service lets you instantiate virtual machines (VM) on demand. When you request the creation of a new VM, you can select the operating systems and the size (RAM, Disk, CPUs) that will be used to run the machine. In this section we will show how to discover which software and sizes are available and how to start a new virtual machine.

3.1. Keypairs

Before attempting to start a new virtual machine, you should have a keypair that will allow you to login into the machine once it is running. Normally you just need to create one keypair that can be reused for all your virtual machines (although you can create as many SSH credentials as you want).

The nova keypair-list command shows your current keypairs. Initially the command should not return anything.

In order to create a new key, use nova keypair-add with a name for the key you want to use redirecting the ouput to the file where you want to store that key. For example, for creating a key named cloudkey that will be stored in cloudkey.pem:

$ nova keypair-add cloudkey > cloudkey.pem

Your recently created keypair should now appear in the list of available keypairs:

$ nova keypair-list
+----------+-------------------------------------------------+
| Name     | Fingerprint                                     |
+----------+-------------------------------------------------+
| cloudkey | 37:fd:b6:73:59:78:fd:f2:7f:e7:9c:1b:9a:88:a5:cb |
+----------+-------------------------------------------------+

Make sure that you keep safe the file cloudkey.pem, since it will contain the private key needed to access your cloud machines. Set proper permissions to the key before using it with chmod 600 clodkey.pem (only user can read or write). If you need to delete one of your keypairs, use the nova keypair-delete command.

3.2. Images

The service lets you run VMs with different Operating Systems, you can list all the available ones with the nova image-list command. The ID of the image will be used as arguments for other commands.

$ nova image-list
+--------------------------------------+----------------------------------------------+--------+--------+
| ID                                   | Name                                         | Status | Server |
+--------------------------------------+----------------------------------------------+--------+--------+
| 6b3046eb-4649-44d6-96c2-9a93d3aab8dc | Fedora 15                                    | ACTIVE |        |
| e803caa2-c247-4088-80fd-54e77b20a5cb | Fedora 15 initrd                             | ACTIVE |        |
| 6823e5b0-13fc-4ce3-afd8-057285820ed2 | Fedora 15 kernel                             | ACTIVE |        |
| 0249a9cc-dced-4c5f-91eb-d6900576206f | Fedora 17                                    | ACTIVE |        |
| f07c936f-7678-40e5-bbfd-f7142a5482ff | Fedora 17 initrd                             | ACTIVE |        |
| a0fbc138-1879-439f-8f78-9b98893778b3 | Fedora 17 kernel                             | ACTIVE |        |
| d3ac534d-d839-4b25-af92-c143930f3694 | Fedora 17 old glibc                          | ACTIVE |        |
| d1eec0f5-e948-435d-899c-d865320698d7 | IFCA Debian Wheezy (2011-08) JeOS            | ACTIVE |        |
| cdbb6f8f-d10e-4e2b-879d-250d29fb9dbb | IFCA Scientific Linux 5.5 + PROOF v5.30.00   | ACTIVE |        |
| 6857ee01-2ba9-4846-b788-9e826dd9aaba | IFCA Scientific Linux 5.5 + ROOT v5.30.00    | ACTIVE |        |
| 18d99a06-c3e5-4157-a0e3-37ec34bdfc24 | IFCA Scientific Linux 5.5 JeOS               | ACTIVE |        |
| 75896bad-05d3-45f6-9958-5940f82d0048 | IFCA Scientific Linux 6.2 JeOS               | ACTIVE |        |
| 486c139e-f34d-465c-959c-1b9c8bf60cfd | IFCA Ubuntu Server 10.04 JeOS                | ACTIVE |        |
| 694f2673-7ea3-4690-a25e-c9dd4297519a | IFCA Ubuntu Server 10.04 JeOS                | ACTIVE |        |
| 66963875-5389-4048-b385-6f7e12a0915f | IFCA Ubuntu Server 11.10 JeOS                | ACTIVE |        |
| 3ef6bb0c-6a17-47c9-a949-70256eb6651e | IFCA openSUSE 11.4 + Compilers               | ACTIVE |        |
| daaed27e-6226-4295-8018-ad3b6b5210f6 | IFCA openSUSE 11.4 + Compilers + Mathematica | ACTIVE |        |
| 29233856-ed8e-4b61-ac81-898eb5e7c263 | IFCA openSUSE 11.4 JeOS                      | ACTIVE |        |
| f4e39219-ad13-495e-a35b-315a94675b0f | Ubuntu 11.10 kernel                          | ACTIVE |        |
| 369455d3-7f84-4630-b60c-e0ebf29a410c | Ubuntu 12.04 JeOS                            | ACTIVE |        |
| 4590d3b0-1df6-49a7-ae68-4dde83089b01 | Ubuntu Server 11.10 JeOS                     | ACTIVE |        |
| fea1838f-29a0-47dd-bd84-6c6cc6806ff3 | cloudpipe                                    | ACTIVE |        |
| 6f02785c-5a39-4e1a-a7e3-75d48f0f0076 | ubuntu 12.04 kernel                          | ACTIVE |        |
+--------------------------------------+----------------------------------------------+--------+--------+

The nova image-show can give you more details about a given image, for example the "IFCA Scientific Linux 5.5 JeOS", which has an ID 18d99a06-c3e5-4157-a0e3-37ec34bdfc24:

$ nova image-show 18d99a06-c3e5-4157-a0e3-37ec34bdfc24
+----------+--------------------------------------+
| Property | Value                                |
+----------+--------------------------------------+
| created  | 2012-01-30T10:12:22Z                 |
| id       | 18d99a06-c3e5-4157-a0e3-37ec34bdfc24 |
| minDisk  | 0                                    |
| minRam   | 0                                    |
| name     | IFCA Scientific Linux 5.5 JeOS       |
| progress | 100                                  |
| status   | ACTIVE                               |
| updated  | 2012-07-18T08:50:48Z                 |
+----------+--------------------------------------+

Information about some of these images is available at Cloud/Images.

3.3. Sizes

As in the case of the image to use, you can select the size of the VM to start. The list of available sizes (flavors in OpenStack terminology) can be obtained with nova flavor-list:

$ nova flavor-list
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+-------------+
| ID | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public | extra_specs |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+-------------+
| 1  | m1.tiny   | 512       | 0    | 0         |      | 1     | 1.0         | N/A       | {}          |
| 2  | m1.small  | 2048      | 10   | 20        |      | 1     | 1.0         | N/A       | {}          |
| 3  | m1.medium | 4096      | 10   | 40        |      | 2     | 1.0         | N/A       | {}          |
| 4  | m1.large  | 7000      | 10   | 80        |      | 4     | 1.0         | N/A       | {}          |
| 5  | m1.xlarge | 14000     | 10   | 160       |      | 8     | 1.0         | N/A       | {}          |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+-------------+

3.4. Starting a machine

For starting a new VM, you need to specify one image, one size and a name for the new machine. Optionally, you can also spcify a keypair (it is always recommended to do so). The nova boot command lets you start the machine. For example, in order to create a VM that:

• runs IFCA Scientific Linux 5.5 JeOS (ID 18d99a06-c3e5-4157-a0e3-37ec34bdfc24)

• of size m1.tiny

• with the key cloudkey

• and named testVM

you would need to issue the following command:

$ nova boot --flavor m1.tiny --image 18d99a06-c3e5-4157-a0e3-37ec34bdfc24 --key_name cloudkey testVM
+------------------------+--------------------------------------+
| Property               | Value                                |
+------------------------+--------------------------------------+
| OS-DCF:diskConfig      | MANUAL                               |
| OS-EXT-STS:power_state | 0                                    |
| OS-EXT-STS:task_state  | scheduling                           |
| OS-EXT-STS:vm_state    | building                             |
| accessIPv4             |                                      |
| accessIPv6             |                                      |
| adminPass              | PGg4KxZo3Fn4                         |
| config_drive           |                                      |
| created                | 2012-09-28T10:02:02Z                 |
| flavor                 | m1.tiny                              |
| hostId                 |                                      |
| id                     | cd9e08b9-6899-4748-909a-2ff667ff1905 |
| image                  | IFCA Scientific Linux 5.5 JeOS       |
| key_name               | cloudkey                             |
| metadata               | {}                                   |
| name                   | testVM                               |
| progress               | 0                                    |
| status                 | BUILD                                |
| tenant_id              | c725b18d8d0643e7b410dc5a8d9ab554     |
| updated                | 2012-09-28T10:02:03Z                 |
| user_id                | db66762e4fe148f8b8484c461a7a7182     |
+------------------------+--------------------------------------+

The id of the machine will allow you to query its status with nova show:

$ nova show cd9e08b9-6899-4748-909a-2ff667ff1905
+------------------------+-----------------------------------------------------------------------+
| Property               | Value                                                                 |
+------------------------+-----------------------------------------------------------------------+
| OS-DCF:diskConfig      | MANUAL                                                                |
| OS-EXT-STS:power_state | 2                                                                     |
| OS-EXT-STS:task_state  | None                                                                  |
| OS-EXT-STS:vm_state    | active                                                                |
| accessIPv4             |                                                                       |
| accessIPv6             |                                                                       |
| config_drive           |                                                                       |
| created                | 2012-09-28T10:02:02Z                                                  |
| flavor                 | m1.tiny (1)                                                           |
| hostId                 | 5ed92271869711d494f1326b9611825d5635ab659ea3e143c13ca8c6              |
| id                     | cd9e08b9-6899-4748-909a-2ff667ff1905                                  |
| image                  | IFCA Scientific Linux 5.5 JeOS (18d99a06-c3e5-4157-a0e3-37ec34bdfc24) |
| key_name               | cloudkey                                                              |
| metadata               | {}                                                                    |
| name                   | testVM                                                                |
| private network        | 172.16.2.8                                                            |
| progress               | 0                                                                     |
| status                 | ACTIVE                                                                |
| tenant_id              | c725b18d8d0643e7b410dc5a8d9ab554                                      |
| updated                | 2012-09-28T10:03:54Z                                                  |
| user_id                | db66762e4fe148f8b8484c461a7a7182                                      |
+------------------------+-----------------------------------------------------------------------+

Alternatively, you can use nova list to get the list of the current machines:

$ nova list
+--------------------------------------+--------------------------------------+--------+-------------------------------------+
| ID                                   | Name                                 | Status | Networks                            |
+--------------------------------------+--------------------------------------+--------+-------------------------------------+
| 4b999e84-b37e-4b95-952d-3414ba271930 | c725b18d8d0643e7b410dc5a8d9ab554-vpn | ACTIVE | private=172.16.2.2                  |
| 3051dea4-0164-4a3a-9af2-14efe7ea93e9 | horizon_test                         | ACTIVE | private=172.16.2.11, 193.146.75.142 |
| e02cee2d-c09f-4429-9724-91d7e10277ec | lbnl                                 | ACTIVE | private=172.16.2.10                 |
| cd9e08b9-6899-4748-909a-2ff667ff1905 | testVM                               | ACTIVE | private=172.16.2.8                  |
+--------------------------------------+--------------------------------------+--------+-------------------------------------+

3.5. Connecting to the machine

Once the machine status is ACTIVE, it will be ready for using it. You can connect via ssh with your key. The IP address of the machine is shown in the nova list output.

$ ssh -i cloudkey.pem root@172.16.2.8
Last login: Mon May 10 16:11:40 2010
[root@testvm ~]#

3.6. VM Lifecycle

Your VM will be available until you explicitly destroy it. You can pause/reboot/resume/delete the machine with these commands:

Deleting the machine will destroy it and the contents of the disk will be lost. Make sure that all your data are stored in a permanent storage before deleting the machine. See the section on volumes for more information.

$ nova delete cd9e08b9-6899-4748-909a-2ff667ff1905
$ nova list
+--------------------------------------+--------------------------------------+--------+-------------------------------------+
| ID                                   | Name                                 | Status | Networks                            |
+--------------------------------------+--------------------------------------+--------+-------------------------------------+
| 4b999e84-b37e-4b95-952d-3414ba271930 | c725b18d8d0643e7b410dc5a8d9ab554-vpn | ACTIVE | private=172.16.2.2                  |
| 3051dea4-0164-4a3a-9af2-14efe7ea93e9 | horizon_test                         | ACTIVE | private=172.16.2.11, 193.146.75.142 |
| e02cee2d-c09f-4429-9724-91d7e10277ec | lbnl                                 | ACTIVE | private=172.16.2.10                 |
+--------------------------------------+--------------------------------------+--------+-------------------------------------+

4. Permanent Storage: Volumes

The VMs use a temporary disk that is destroy when the machine is deleted. If you need permanent storage for your data, you can use Volumes. Volumes are raw block devices that can be created dynamically with a desired size. Volumes can be attached and detached from a running cloud VM to be used as a data disk (similarly to a usb stick that can be plug and unplugged to a computer).

4.1. Creating Volumes

The nova volume-create creates new volumes. You must specify the size (in GB) and optionally a name. In our case we will create a new volume with 5GB called mydata

$ nova volume-create --display-name 'mydata' 5
+---------------------+----------------------------+
| Property            | Value                      |
+---------------------+----------------------------+
| attachments         | []                         |
| availability_zone   | nova                       |
| created_at          | 2012-09-28 15:16:44.590600 |
| display_description | None                       |
| display_name        | mydata                     |
| id                  | 14                         |
| metadata            | {}                         |
| size                | 5                          |
| snapshot_id         | None                       |
| status              | creating                   |
| volume_type         | None                       |
+---------------------+----------------------------+

The nova volume-list shows all available volumes:

$ nova volume-list
+----+-----------+--------------+------+-------------+-------------+
| ID | Status    | Display Name | Size | Volume Type | Attached to |
+----+-----------+--------------+------+-------------+-------------+
| 14 | available | mydata       | 5    | None        |             |
+----+-----------+--------------+------+-------------+-------------+

The volume is now created and can be attached to a VM.

4.2. Attaching Volumes

Attaching is the process of associating a volume with a given instance, so the volume is seen as a new block device in the VM. The command to attach the volume is nova volume-attach, and the parameters are:

• the id of the VM
• the id of the volume

• the local block device where the volume will be attached. These devices are in the form /dev/xvd<DEVICE_LETTER>, where <DEVICE_LETTER> goes from c to z (/dev/xvdc, /dev/xvdd, ..., /dev/xvdz)

For example:

$ nova volume-attach  9f870141-638e-4eb9-a2fa-ec770d1edb79 14 /dev/xvdc
+----------+-------+
| Property | Value |
+----------+-------+
| id       | 14    |
| volumeId | 14    |
+----------+-------+

nova volume-list should now show that the volume is attached:

$ nova volume-list
+----+--------+--------------+------+-------------+--------------------------------------+
| ID | Status | Display Name | Size | Volume Type | Attached to                          |
+----+--------+--------------+------+-------------+--------------------------------------+
| 14 | in-use | mydata       | 5    | None        | 9f870141-638e-4eb9-a2fa-ec770d1edb79 |
+----+--------+--------------+------+-------------+--------------------------------------+

Log into your VM and check with dmesg that the volume is now attached:

[root@testvm ~]# fdisk -l | grep Disk
Disk /dev/xvda doesn't contain a valid partition table
Disk /dev/xvdc doesn't contain a valid partition table
Disk /dev/xvda: 3220 MB, 3220176896 bytes
Disk identifier: 0x00000000
Disk /dev/xvdc: 5368 MB, 5368709120 bytes
Disk identifier: 0x00000000

Now you can mount your volume (for example in /srv) and start using it:

[root@testvm ~]# mount -t ext4 -o sync /dev/xvdc /srv
[root@testvm ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda             3.0G  790M  2.1G  28% /
tmpfs                 245M     0  245M   0% /dev/shm
/dev/xvdc             5.0G  138M  4.6G   3% /srv

4.3. Detaching Volumes

Once you are done with the volume, you can umount it at your VM:

[root@testvm ~]# umount /dev/xvdc

And detach it from the VM with nova detach with the VM and Volume id as arguments:

$ nova volume-detach  9f870141-638e-4eb9-a2fa-ec770d1edb79  14

The volume should appear again as available when you list it:

$ nova volume-list
+----+-----------+--------------+------+-------------+-------------+
| ID | Status    | Display Name | Size | Volume Type | Attached to |
+----+-----------+--------------+------+-------------+-------------+
| 14 | available | mydata       | 5    | None        |             |
+----+-----------+--------------+------+-------------+-------------+

4.4. Deleting volumes

You can reuse as many times as you like your volumes in your VMs, the data stored in them will persist after you have destroy your VMs. If you no longer need one of your volumes, you can do so with the nova volume-delete command. Once you delete a volume, you will not be able to access to its data again!.

$ nova-delete 9

5. Networking

All created VMs have a private IP within a VLAN accessible only from the VMs. If you need access to a machine from outside that VLAN, there are two alternatives: using a VPN or assigning public IPs to the VMs.

5.1. VPN

VPNs are currently being tested in the infrastructure. The documentation will be updated as soon as the features are available.

5.2. Public IPs

IFCA provides a pool of public IPs to use them in the cloud service. These can be allocated for your use and assigned to your VMs. Please note that the number of public IPs is limited!

New IPs are created with nova floating-ip-create:

$ nova floating-ip-create
+----------------+-------------+----------+------+
| Ip             | Instance Id | Fixed Ip | Pool |
+----------------+-------------+----------+------+
| 193.146.75.142 | None        | None     | nova |
+----------------+-------------+----------+------+

You can get the list of the current available IPs with nova floation-ip-list:

$ nova floating-ip-list
+----------------+-------------+----------+------+
| Ip             | Instance Id | Fixed Ip | Pool |
+----------------+-------------+----------+------+
| 193.146.75.142 | None        | None     | nova |
+----------------+-------------+----------+------+

This newly allocated IP can now be associated to a running VM with nova add-floating-ip <VM ID> <IP>:

$ nova add-floating-ip  9f870141-638e-4eb9-a2fa-ec770d1edb79 193.146.75.142

The list command will show that the IP is assigned:

$ nova floating-ip-list
+----------------+--------------------------------------+------------+------+
| Ip             | Instance Id                          | Fixed Ip   | Pool |
+----------------+--------------------------------------+------------+------+
| 193.146.75.142 | 9f870141-638e-4eb9-a2fa-ec770d1edb79 | 172.16.2.9 | nova |
+----------------+--------------------------------------+------------+------+

And you will be able to connect to the machine with this new IP:

$ ssh -i cloudkey.pem root@193.146.75.142
The authenticity of host '193.146.75.142 (193.146.75.142)' can't be established.
RSA key fingerprint is 29:80:9b:28:e7:8a:00:fe:6c:60:ef:e6:a6:71:33:bd.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '193.146.75.142' (RSA) to the list of known hosts.
Last login: Fri Sep 28 16:54:40 2012 from gridui02.ifca.es
[root@testvm ~]#

IPs can be disassociated from the VM with the nova remove-floating-ip <VM ID> <IP>:

$ nova remove-floating-ip 9f870141-638e-4eb9-a2fa-ec770d1edb79 193.146.75.142

IPs can be associated/disassociated as many times as needed to any VM you may have running. When no longer needed, release the IP address (i.e. it will no longer be available four your use):

$ nova floating-ip-delete 193.146.75.142

6. Using the web portal

The OpenStack dashboard lets you perform all the operations described in this manual from your web browser.

6.1. Creating a Machine with OpenStack

Go to http://portal.cloud.ifca.es to access to IFCA OpenStack.

6.2. Image and size selection

You should launch the image that you want to use (within a list of OS) and click “Launch”. A new popup window will be shown and you have to choose the configuration of the system (requirements, name of the server...).

6.3. Create SSH credentials

You must import or create a new key in order to access to that image. To do so go to “Access & Security” tab and click on Create or Import Keypair.

6.4. Connect to the server

In order to access throw ssh to the image, you must asign an IP to the instance. Click on “Access & Security” again and select “Allocate IP to project”. Choose the type of IP that you want to use and click “Allocate IP”. After that, you need to link that IP with your new image. Click on the button “Associate IP” of your new IP and select the instance that you have just created.

6.4.1. SSH Connection

Last step is to download the keypair that you have created or imported and move it to the machine that you will use to conect to the instance. Change permission to 600 and use the following command to connect:

$ ssh -i clave.pem root@<cloud.image.IP>

Done

6.5. VM Lifecycle

You can access your VM until you destroy it. Management is allowed thorugh some actions that you can see below: