FINKI Cloud - User Guide

Document Version Log

1. Introduction

1.1 Scope and Purpose

The FINKI Cloud is a cloud service provided by the University Ss. Cyril and Methodius, Faculty of Computer science and engineering in Skopje.

The infrastructure is based on OpenStack and is hosted on 15 Huawei servers, each with 128GB RAM and 20 HT CPU cores, totalling in 300 vCPU cores and 37TB SSD and 32 TB SAS storage. The system is in production from 2017 as a National cloud system. The connectivity to the Internet is 1Gbit through MARNET provided link to GEANT.

Currently the system hosts templates for all popular Linux distributions, and Windows variations.

The target user communities for this service are researchers and research groups form all scientific areas, with special focus on the long tail of science, due to the modest capacity of the services. It is meant to provide those researchers and groups easy access to computing environment in the cloud, with images for the operating systems and solutions most frequently used in scientific cloud environments.

1.2 Organization of the User Guide

This user guide is organized in multiple sections, each explaining different aspects of the cloud platform, giving the end-users advice on recommended actions, and troubleshooting steps that can be performed. The rest of the document is organized as follows:

In section two a general overview of the requirements for accessing the service and the login procedure is given, focusing on the steps that need to be taken by a federated user to use it for the first time.

Section three documents the most common workloads that a user might need to perform in order to get a fully functioning virtual machine with both inbound and outbound network access, along with steps required for attaching multiple persistent storage disks, assigning flexible floating IP addresses, and tips for securing inbound and outbound access to the instances themselves. The section is concluded with topics focused on working with user identity, user management and public key management.

Section four deals with the most common troubleshooting scenarios that a user might perform by themselves and gives additional instructions on contacting the Helpdesk, as well as checking the operational status of the service at any point in time.

2. Accessing the Service

2.1 Requirements

To access the Web based graphical user interface of this service, the user has to use a modern and standards compliant Web browser with enabled support for JavaScript, such as the latest supported versions of Google Chrome, Mozilla Firefox, Apple Safari or Microsoft Edge.

To access the REST API endpoints of this service, the user has to be able to execute standard HTTP methods, including HTTPS (TLS 1.2). Additionally, open-source OpenStack API libraries are available for the majority of operating systems and programming languages, further easing this interaction.

Users must be verified by one of the partners in the project and granted access by the relevant Work Package leader or Task Group leader before accessing the service.

2.2 Login Procedure

The authentication procedure can be performed through the Web based UI (Horizon) or through the command line interface, utilizing the REST API of the service. The Web based interface provides support both for a username-password pair (Figure 1) and for the Project’s AAI - NI4OS AAI (Figure 2).

3. Documentation of Modules and/or Workflows

This section explains the most common actions performed by end-users and provides suggestions on the recommended configuration options to be used during initial resource deployment

3.1 Working With Virtual Machine Instances

Upon successful login using one of the login methods explained in section 2, the user is presented with an overview dashboard showing the most important data including the usage of the allocated compute, storage, and networking resources (Figure 3). The user has the option to access the active instances directly from the list available at the bottom of the view or by selecting the menu option Instances within the Compute group on the left side of the screen (Figure 4).

3.1.1 Creating a New Virtual Machine Instance

The creation of a new virtual machine instance is one of the most common workflows that can be undertaken by an authorized user. The process consists of multiple steps that must be completed before the instance is assigned to a compute node and provisioned.

The first step towards launching a new virtual machine instance is going to the instances section by using the navigation menu on the left-hand side and choosing Compute -> Instances. There, in the top right corner a Launch Instance button is visible (Figure 4) which opens the virtual machine creation wizard (Figure 5).

This wizard consists of multiple steps, outlined on the left-hand side of the pop-up window, each gathering different information for the instance to be created.

1. In the Details step (Figure 5) enter a name for your new instance along with an optional description of its purpose in the Description field. You can automatically replicate the entered configuration options and create multiple instances at once by increasing the value in the Count field.

2. In the Source step (Figure 6), you have the possibility of selecting the operating system of the new instance, along with adding any additional volumes besides the default one, where the operating itself will be installed. In the Select Boot Source dropdown menu select Image, after which an exhaustive list of operating systems will appear. From the list select the desired option and move it to the Allocated section by clicking on the upwards facing arrow. In this step, you also have the possibility to choose if you would like to add an additional block volume, optionally making it independent from your instance (e.g. deleting the instance will not delete the additional block volume). To do so, choose Yes for the Create New Volume option and enter the desired volume size, taking into account your project’s storage quota visible on the Overview page (Figure 3).

3. In the Flavor step you have the option of selecting the compute, memory, and storage capacity of the instance. Note that the storage capacity only applies to the main disk that will house the operating system. There are a number of preconfigured flavors that you can choose from, offering balanced configurations of CPU, memory, and storage space, suitable for the most common workloads (Figure 7).

4. The Networks step allows the selection of the virtual networks that the virtual machine will be part of. Each selected network will be represented by a different network interface inside the virtual machine. Take great care when adding network interfaces during the initial deployment process of a virtual machine - the start-up script of your chosen operating system might select a non-optimal default interface, thus rendering your instance inaccessible from the outside. For public access through a publicly routable IP address, select the option external-3000, as shown in figure 8.

5. Since the step Network Ports does not contain any required fields, it can be skipped completely.
6. The Security Groups tab does not contain any required fields, but its configuration is essential for uninterrupted access to your virtual machine and its hosted services. In this step you have the possibility of assigning a set of predefined firewall rules to your virtual machine which will control both inbound and outbound traffic. The default option might not be suitable in most cases, since it offers only inbound SSH access, and might need to be customized according to the needs of the user. However, rule changes can be done after the deployment of the virtual machine, without requiring a system restart. More information is available in the section 3.3.4 - Creating and altering security groups.
7. The final step that is essential for a deployment of a functional virtual machine instance is Key Pair. This step allows the user to select a single SSH public key that will be automatically added as an authorized key inside the operating system. A new SSH public key can be imported to the service at any point in time for future use by utilizing the option Import Key Pair. The pasted public key should be in the common OpenSSH format.

8. Once all the above steps have been completed, the instance can be deployed by clicking on the blue Launch Instance button in the bottom right of the pop-up window.

The virtual machine will go through multiple phases, such as Allocating Block Storage, Spawning, etc… Please patiently wait until it reaches the Active state as seen in figure 9. Once in the Active state you can use the default remote access protocol (SSH for GNU/Linux instances, Remote Desktop for Microsoft Windows instances) for the selected operating system for connecting to the virtual machine by using its allocated IP address. If a public key has been selected during the deployment process, then no password will be required for login. The username used will depend on the operating system that has been requested, and it is usually the name of the distribution (ubuntu for Ubuntu instances, debian for Debian instances, centos for CentOS instances…).

3.1.2 Interacting With an Existing Virtual Machine Instance

Additional configuration can be performed to any virtual machine after it has been deployed. To do so, navigate to the Compute -> Instances section of the Web interface, where you are presented with a list of all the deployed instances in your current project, as seen in figure 9. Upon selecting a desired instance from this list, the user is presented with a detailed view of the instance, including the option to perform one of the available actions, from managing the network settings and attached interfaces, working with volumes and snapshots, to controlling the execution of the instance including pausing, suspending or shutting down the instance (Figure 10). Many of the functions are also available through the tabbed User Interface with Overview being the tab selected by default, while Interfaces, Log, Console and Action Log provide further information and management options to the user (Figure 11, Figure 12).

3.2 Working With Volumes and Snapshots

All volumes defined inside the given project can be managed from the Volumes -> Volumes section and the other sections on the same level (Snapshots, Groups, Group Snapshots).

3.2.1 Attaching a New volume to an Existing Virtual Machine Instance

A new volume attachable to any existing virtual machine can be created by selecting the Create Volume button in the top right-hand side of the Volumes -> Volumes section (Figure 13). Once clicked, a popup window appears where the user can select the desired configuration for the new volume, as shown in figure 14.

Start by selecting the desired name for the new volume through which it will be identified. Choose No source, empty volume in the Volume Source dropdown menu. The Type of the volume depends on whether you require high disk performance (RBDFAST - storage backed by SSD disks) or normal performance (RBD - storage backed by mechanical disks configured in RAID10). Finally, select the desired size of the new disk in gigabytes, taking into account your project’s storage quota.

Once the volume has been created, it can be attached to any running instance while running. For more information please see section 3.1.2, Interacting with an existing virtual machine instance.

3.2.2 Using Volume Snapshots

Any volume can be snapshotted and later this snapshot can be used as a source volume for a new virtual machine instance.

The two main ways in which a snapshot can be created are:

1. Through the instance overview page in Compute -> Instances, by clicking on the dedicated Create Snapshot button (Figure 9) and providing a Snapshot Name.
2. Through the Volumes overview page in Volumes -> Volumes and selecting Create Snapshot from the expanded right-hand side dropdown control (Figure 15).

All the created snapshots for a given volume are listed in the Snapshots tab of the Volumes section, as presented in figure 16. The details about a particular snapshot can be seen by clicking on its name, opening an interface similar to the one presented in figure 17.

After a snapshot has been created a new instance can be created from it by selecting Instance Snapshot as the boot source during the instance creation procedure, more details are available in section 3.1.1, Creating a new virtual machine instance.

3.3 Working With Networks

Networking is an essential part of the virtual machine instances configuration. In this section, a brief overview of this topic is given, along with short guides on how to assign a public IP address to a given instance, utilize private networks, and create floating IPs that can be freely moved between virtual machines.

3.3.1 Public vs. private networking

Depending on how the virtual machine will be used, users have the option to allocate and associate a public IP address with it. To do so, as stated in section 3.1.1 - Creating a new virtual machine instance, the global network that is accessible to all projects, external-3000, must be selected. In cases when no public access is needed, or users have the opportunity to use a jump or bastion host to access their resources, internal per-project networks can be used, explained in more details below.

3.3.2 Creating Internal Project Networks

Internal networks allow you to completely isolate your virtual machines from other projects and provide secure means of intra-project VM communication. In order to create a new internal network, the following steps must be executed:

1. Navigate to the Project -> Network -> Networks page by utilizing the left-hand side navigation menu. You will be presented with a list of the global networks, as well as all existing project networks (Figure 18).

2. Click on the Create Network button to open the new network wizard. Notice that this wizard contains multiple subsections, each placed in a dedicated tab available at the top of the pop-up (Figure 19).

3. In the Network tab of the new network wizard, specify only the name for the new network and leave the other settings to their default values:

   1. Enable Admin State - checked
   2. Shared - Unchecked
   3. Create Subnet - Checked

4. In the Subnet tab of the new network wizard, enter the name for the first subnet created inside this network, along with the address range that will be dedicated to it using a CIDR prefix notation in the Network Address field (e.g. 192.16.1.0/24). If this will solely be an internal, air-gapped network, with no external Internet access, select Disable gateway, otherwise, if you want to configure NAT for external Internet access for resources in this network, leave the default values (Figure 20):

   1. Gateway IP - empty
   2. Disable Gateway - unchecked

5. In the final tab, Subnet Details, check whether you would like to enable DHCP for automatic network configuration within this virtual network, and if so set the default DHCP pool, along with the default DNS name servers and any additional host routes to be pushed to the VMs inside this network (Figure 21).

Once all of the required fields have been configured, the network can be created using the blue Create button in the right-hand side corner.

If a NAT configuration is required, where instances added to the newly created network would also have public outbound access, then additional configuration is required - a router in the Project -> Network -> Routers section will need to be configured. The required steps for this are:

1. Once the Project -> Network -> Routers section has been opened, click on the Create Router button in the top right-hand corner. A pop-up window like the one in figure 22 will appear.

2. Enter a name for the new router and from the External Network dropdown select the external-3000 option, keeping the option Enable SNAT option checked. This will allocate one public IP address from the external-3000 network as a gateway address.
3. Click on the blue Create Router button in the bottom-right corner to save the new configuration.
4. Once the router has been created, open its detailed view by clicking its name on the overview page in the same section.
5. Navigate to the Interfaces tab (Figure 23) and click on the Add Interface button in the right-hand corner. Select your previously created subnet from the Subnet dropdown list, leaving the IP Address field unpopulated.

Once all of the above steps have been completed, any instance that has been added to the newly created network will have unrestricted outbound Internet access.

3.3.3 Allocating and Assigning a Floating IP Address to a Virtual Machine

Floating IPs are IP addresses that are not tied to any particular virtual machine and can be reassigned from one instance to another, using just a few clicks. This allows you to keep your known public IP addresses, even when the underlying virtual machines will be deleted.

Floating IPs can be provisioned from the Project -> Network -> Floating IPs page accessible through the left-hand side navigation menu. The steps that need to be taken are the following:

1. After opening the Floating IPs details page, you will be presented with a list of all associated floating IPs to your current project. Click on the top right-hand side button titled Allocate IP to Project.
2. Give a short name to your new floating IP through which you will be able to later identify it and select external-3000 in the dropdown field titled Pool.
3. After the IP has been allocated your project, it can be associated with any existing virtual machine instance, by clicking on the Associate button.
4. In the pop-up menu that appears, select an existing network interface from one of the virtual machine instances and finalize the process by clicking on the blue Associate button in the bottom right corner.

3.3.4 Creating and Altering Security Groups

The purpose of a security group is to regulate inbound and outbound network traffic to a virtual machine. Every project has a rather restrictive security group defined by default and in many cases changes to the default security group or a creation of a completely new security group might be required.

Security groups can be defined from the Project -> Network -> Security Groups menu. To do so, the following steps need to be followed:

1. Click on the top right-hand button titled Create Security Group. Enter a name and an optional description for the new security group being created.
2. After the security group has been created, it will appear in the overview list with all the other existing security groups (Figure 24). Additional rules can be added and removed by clicking on the Manage Rules button next to the security group.

3. After clicking the Manage Rules button, an overview page appears listing all of the currently associated rules with the security group. A new rule can be added by clicking on the Add Rule button in the top right-hand corner.
4. A new pop-up window appears containing multiple fields relating to the nature of the new rule that needs to be added (Fig 25). Select the protocol that the rule applies to, any custom ports, the direction of traffic and the remote addresses to which this rule applies.

5. Once created, the new security rule is automatically applied to any virtual machine instances using the given security group.

3.4 Working With User Identity

This part of the guide focuses on how end-users can interact with other members of their project and alter their public SSH keys associated with their profile.

3.4.1 Previewing the List of Users Assigned to a Project

A single project in FINKI Cloud can be accessed by multiple members, allowing collaboration and sharing of the resource quota assigned to that particular project.

To view the list of users assigned to a given project, users can navigate to Users page accessible through the navigation menu on the left-hand side, Identity -> Users.

3.4.2 Using SSH Keys

As discussed previously in section 3.1.1 assigning a valid SSH public key is an essential step during the deployment of a virtual machine instance. Users can preview all their associated SSH public keys and even create new SSH key pairs by accessing the Project -> Compute -> Key Pairs page through the left-hand side navigation menu. There, either an existing public key can be imported, by clicking on the Import Public Key button, or a complete new private key pair can be generated by the system itself, automatically associating the public key part with the users profile and providing a download link for the private key. A new private key pair can be generated by clicking on the Create Key Pair button.

4 Troubleshooting

In case of issues with the FINKI cloud service itself, or with some of the resources provisioned through it, users should first check the status of the service through the NI4OS Europe monitoring dashboard, available at: https://argo.ni4os.eu/.

Should there be no known issues with the status of the service, users can attempt to troubleshoot some common problems themselves, by following the steps explained in the subsections below.

If no solution can be found and the problem persists, end-users can submit a help request through the helpdesk, available at: https://helpdesk.ni4os.eu/

4.1 Dealing With Failures During Instance Deployment

In case a deployment of a new virtual machine fails, its status will be changed to Failed. This usually indicates a problem with the underlying infrastructure, but end-users can reattempt the provisioning of the virtual machine by deleting the failed instance and creating a new one.

4.2 Connectivity Problems From/To Virtual Machine

If after a successful virtual machine deployment there is a problem accessing it remotely or some ports that are supposed to be open are not reachable, then it is most likely a misconfiguration of the security rules associated to the security group assigned to the given virtual machine. Please refer to section 3.3.4 for more details on how to configure inbound and outbound network policies.

4.3 Virtual Machine Authentication Failures

After a virtual machine deployment, remote access to it might not be possible. To troubleshoot the issue, please verify the following:

1. Is the port through which you are trying to connect to the virtual machine open (e.g. port 22 for SSH, port 3389 for RDP)? This can be tested with any networking tool that supports opening TCP connections to a given port, for example telnet or netcat. If the port is not open, refer to the troubleshooting section for connectivity problems, available above.
2. If the port is open, but an Authentication Failure message is received, please check whether the correct SSH public key has been assigned to the virtual machine by going to the details view for that particular instance and scrolling to the Metadata subsection. The Key Name field should state the name of the public key that has been authorized for this instance.

5 Integration With the Project Infrastructure and Services

The service will be fully integrated with the NI4OS-Europe and EOSC infrastructure, providing AAI, monitoring, accounting and helpdesk support. More about the service can be found at the service catalog available at: https://catalogue.ni4os.eu/.