Pacific National Exhibition, known by Vancouver and BC locals as “The PNE”, is a family friendly Vancouver staple that has been around for over 100 years. The PNE is host to Playland, an amusement park, The Fair, a two-week long event focused on Playland, Fright Nights haunted houses over Halloween, and many shows, concerts, conferences, and other major events at several venues on the sprawling property that includes park land and sports fields.
When the PNE first joined IONICA, quite a number of years ago, maintaining site and service availability was a challenge, especially during major events. Our solution: build a horizontally and vertically scalable, fault-tolerant environment — with load balanced web services, a database cluster, and comprehensive 24/7/365 monitoring and observability.
In this article, I will show you how the PNE leverages IONICA site reliability engineering expertise delivered on OVHcloud Public Cloud. We’ll break down how we use the OVHcloud OpenStack API CLI to scale up PNE server instances at OVHcloud Public cloud to meet demand exceeding 10,000 concurrent requests, and then scale them back down during normal traffic levels to save costs and reduce environmental impact.
Prerequisites
First, let’s install the OpenStack client utilities. We will use them to interact with the APIs. You can later use them to script or otherwise automate tasks undertaken with the API CLI. The tasks I’m about to show you can also be done on the OVHcloud OpenStack Horizon web interface. The web interface is pretty straightforward to use, but if it would be helpful to have a blog post about that in the future, let us know!
If you are running Debian or Ubuntu on your workstation:
apt install python3-openstackclient python3-novaclient python3-cinderclient python3-swiftclientIf you are running Fedora, Rocky, Alma, Oracle, or RHEL on your workstation:
yum install python3-openstackclient python3-novaclient python3-cinderclient python3-swiftclient
Create OpenStack users for your OVHcloud Public Cloud project
- Log in to your OVHcloud account dashboard at https://ca.ovh.com/manager/.
- Click the “Public Cloud” menu link at the top of the page.
- Select the project you would like to administer with the OpenStack APIs.
- In the left pane of the dashboard, scroll to the bottom, then click “Users and Roles“.
- Click the “Add User” button at the top of the page.
- Type a description for the user in the “User description” field, click “Next”.
- Tick the “Administrator” box.
- A user name, like “user-JwqZJB9Dncim” will be generated, and this user will be listed on the users page. A password for this user will appear at the top of that list. Copy that password and store it in a secure password manager. After 30 – 60 seconds, refresh the users page in the dashboard — the status of the user should then change from “Creating” to “Enabled”.
- If you forget to copy the password before reloading the users page, don’t worry. You can easily reset it by clicking the ‘…’ button at the very right edge of the page, next to the status column in the user list. You can also generate a token for users from that same options menu.
Get and invoke the source file for your OpenStack project
- Click the ‘…’ menu, then click “Download OpenStack RC file. Be sure to select a file for the exact data centre location the machines you are working with are located in.
- Ensure the openrc file includes the proper data centre location. For example, the file below had
OS_REGION_NAME(this variable is located near the bottom of the openrc file) set to"BHS", but the correct region name is"BHS5". If you are unsure exactly which region your public cloud instances are in, click “Instances” in the left pane of the OVHcloud Public Cloud dashboard. The region name is printed in brackets next to the location name (which is the second column from the left). - If you would like to interact with instances or resources in different regions, but in the same project, you may need to export the
OS_REGION_NAMEvariable for each region in your project before interacting with resources in each other region, putting the relevant region names as values for those variables. - After downloading, I rename each openrc file according to its project name. So if I had a project called, “Example”, I would name its openrc file
example-openrc.sh. - Invoke the OpenStack configuration by typing
'source filename.sh'in your local shell.
Display OpenStack resources to test API connectivity and set-up
To test our set-up, and to display the instances we are working with, run openstack server list in your local environment, after invoking your openrc source file. If a list of servers that are currently running in your project does not appear, check your openrc file and command syntax, and verify you have all the client-side prerequisites installed and functional. Perhaps the most likely issue is that the region name is not correct, as mentioned above.
Instance types
Printing a list of available VM types and sizes is simple (note that the command uses the US English spelling of “flavour”):
openstack flavor list
The output looks like this:
I’ve truncated that list in the screen-shot, since it is quite long. As of the time of this writing, there are 82 instance types to choose from, including various sizes of each. OVHcloud handily uses descriptive names for each type.
A quick explanation of the various instance types or “flavours”:
| Name Starts With | Instance Type / Class |
| b2 | General purpose |
| c2 | High-frequency CPU (above 3 Ghz) |
| r2 | Memory intensive |
| i2 | Fast I/O (IOPS) |
| t2 | Graphics processing intensive |
‘win’ is appended to the names of instance types running the Windows Server operating system.
‘flex’ is appended to the names of instance types with “flex” storage.
More information on OVHcloud instance types can be found on the pricing page.
Flex instances — those with that ‘-flex’ bit appended to their names — and storage are particularly important in cases where instances need to be scaled up and down with any frequency. That’s because with OpenStack — and indeed with the vast majority of disk partitioning schemes — storage can not easily be changed on a given instance, due to complications with shrinking partitions while maintaining data integrity. To easily work around this, OVH offers flex instances, which have a fixed primary disk size — they all come with a 50 GB primary disk, no matter which flex instance you choose. Other instance types can only be scaled up, but not back down. So if you expect to have to scale a given instance up and down over its life time, choose a flex instance type when provisioning it. If you did not use a flex image at provision time, you can convert an instance to a flex instance, as long as it is a “smaller” instance type before doing so.
Make a back-up
It’s highly unlikely this resize operation will go wrong, and OpenStack has an effective automatic roll-back mechanism. That said, it is crucial to always make back-ups of important machines and data before making substantial changes to them, where the machine or data could potentially be lost.
It’s not required, but depending on the circumstances, it may be wise to also gracefully shut down some services before beginning the resize operation. A resize job in OpenStack does stop the instance, and then boot it back up. So now is also a good time to make sure essential services are set to start at boot, and any additional filesystem mounts are properly in /etc/fstab on Linux instances (or that you at least know how to mount them again after boot).
Instructions on backing up instances in OVHcloud:
- Documentation on backing up instances with the OVHcloud Horizon OpenStack web interface.
- OVHcloud blog post on backing up instances with the OpenStack CLI.
Resizing an instance
It’s almost time to actually resize an instance! Before we do that, however, we need to know two things:
- The exact name of the particular instance we want to change.
- the ID number of the instance type or flavour we want to change that instance to.
openstack server list
openstack flavor listLet’s also fetch the details of the instance we’re resizing, so that we can compare them with that of the instance after the resize task is complete:
openstack server show [instance_ID]
Now (optionally) stop and resize the instance (where ‘test_instance is the name of the instance to be resized, and the instance type to resize to is b2-7-flex). Stopping the instance is not absolutely required, but good practise to ensure that all data is flushed to disk, and it is potentially the safest way to perform the resize task.
openstack server stop test_instance
openstack server resize --flavor 1929fd88-3c93-4011-b0bb-cc2710eab476 test_instance
The resize operation and verification can take several minutes. You can check to see if it is complete by running openstack server show every so often. The status field near the bottom will say, “RESIZING” while the resize operation is in progress, and it will report “ACTIVE” when everything is all done and the instance is back in action.
Done!
The procedure for scaling an image back down is exactly the same as for scaling it up — just enter a different ID for the ‘–flavor’ argument.
If you would prefer to resize instances with a web interface, instead of a CLI, OVHcloud has concise and easy-to-follow documentation on how to do it that way.
I hope you found this post informative and fun! A great way to take the OVHcloud OpenStack CLI further is to automate everything we’ve done in your favourite scripting language.
