Scaling up the PNE for Massive Traffic Demand



Generic chart of rising figures.

Pacific National Exhibition, known by Vancouver and BC locals as “The PNE”, is a family friendly Vancouver staple that has been around for over 100 years. The PNE is host to Playland, an amusement park, The Fair, a two-week long event focused on Playland, Fright Nights haunted houses over Halloween, and many shows, concerts, conferences, and other major events at several venues on the sprawling property that includes park land and sports fields.

When the PNE first joined IONICA, quite a number of years ago, maintaining site and service availability was a challenge, especially during major events. Our solution: build a horizontally and vertically scalable, fault-tolerant environment — with load balanced web services, a database cluster, and comprehensive 24/7/365 monitoring and observability.

In this article, I will show you how the PNE leverages IONICA site reliability engineering expertise delivered on OVHcloud Public Cloud. We’ll break down how we use the OVHcloud OpenStack API CLI to scale up PNE server instances at OVHcloud Public cloud to meet demand exceeding 10,000 concurrent requests, and then scale them back down during normal traffic levels to save costs and reduce environmental impact.


First, let’s install the OpenStack client utilities. We will use them to interact with the APIs. You can later use them to script or otherwise automate tasks undertaken with the API CLI. The tasks I’m about to show you can also be done on the OVHcloud OpenStack Horizon web interface. The web interface is pretty straightforward to use, but if it would be helpful to have a blog post about that in the future, let us know!

If you are running Debian or Ubuntu on your workstation:

apt install python3-openstackclient python3-novaclient python3-cinderclient python3-swiftclient

If you are running Fedora, Rocky, Alma, Oracle, or RHEL on your workstation:

yum install python3-openstackclient python3-novaclient python3-cinderclient python3-swiftclient

Create OpenStack users for your OVHcloud Public Cloud project

Get and invoke the source file for your OpenStack project

Display OpenStack resources to test API connectivity and set-up

To test our set-up, and to display the instances we are working with, run openstack server list in your local environment, after invoking your openrc source file. If a list of servers that are currently running in your project does not appear, check your openrc file and command syntax, and verify you have all the client-side prerequisites installed and functional. Perhaps the most likely issue is that the region name is not correct, as mentioned above.

Instance types

Printing a list of available VM types and sizes is simple (note that the command uses the US English spelling of “flavour”):

openstack flavor list

The output looks like this:

I’ve truncated that list in the screen-shot, since it is quite long. As of the time of this writing, there are 82 instance types to choose from, including various sizes of each. OVHcloud handily uses descriptive names for each type.

A quick explanation of the various instance types or “flavours”:

Name Starts WithInstance Type / Class
b2General purpose
c2High-frequency CPU (above 3 Ghz)
r2Memory intensive
i2Fast I/O (IOPS)
t2Graphics processing intensive

‘win’ is appended to the names of instance types running the Windows Server operating system.

‘flex’ is appended to the names of instance types with “flex” storage.

More information on OVHcloud instance types can be found on the pricing page.

Flex instances — those with that ‘-flex’ bit appended to their names — and storage are particularly important in cases where instances need to be scaled up and down with any frequency. That’s because with OpenStack — and indeed with the vast majority of disk partitioning schemes — storage can not easily be changed on a given instance, due to complications with shrinking partitions while maintaining data integrity. To easily work around this, OVH offers flex instances, which have a fixed primary disk size — they all come with a 50 GB primary disk, no matter which flex instance you choose. Other instance types can only be scaled up, but not back down. So if you expect to have to scale a given instance up and down over its life time, choose a flex instance type when provisioning it. If you did not use a flex image at provision time, you can convert an instance to a flex instance, as long as it is a “smaller” instance type before doing so.

Make a back-up

It’s highly unlikely this resize operation will go wrong, and OpenStack has an effective automatic roll-back mechanism. That said, it is crucial to always make back-ups of important machines and data before making substantial changes to them, where the machine or data could potentially be lost.

It’s not required, but depending on the circumstances, it may be wise to also gracefully shut down some services before beginning the resize operation. A resize job in OpenStack does stop the instance, and then boot it back up. So now is also a good time to make sure essential services are set to start at boot, and any additional filesystem mounts are properly in /etc/fstab on Linux instances (or that you at least know how to mount them again after boot).

Instructions on backing up instances in OVHcloud:

Resizing an instance

It’s almost time to actually resize an instance! Before we do that, however, we need to know two things:

openstack server list
openstack flavor list

Let’s also fetch the details of the instance we’re resizing, so that we can compare them with that of the instance after the resize task is complete:

openstack server show [instance_ID]

Now (optionally) stop and resize the instance (where ‘test_instance is the name of the instance to be resized, and the instance type to resize to is b2-7-flex). Stopping the instance is not absolutely required, but good practise to ensure that all data is flushed to disk, and it is potentially the safest way to perform the resize task.

openstack server stop test_instance
openstack server resize --flavor 1929fd88-3c93-4011-b0bb-cc2710eab476 test_instance

The resize operation and verification can take several minutes. You can check to see if it is complete by running openstack server show every so often. The status field near the bottom will say, “RESIZING” while the resize operation is in progress, and it will report “ACTIVE” when everything is all done and the instance is back in action.


The procedure for scaling an image back down is exactly the same as for scaling it up — just enter a different ID for the ‘–flavor’ argument.

If you would prefer to resize instances with a web interface, instead of a CLI, OVHcloud has concise and easy-to-follow documentation on how to do it that way.

I hope you found this post informative and fun! A great way to take the OVHcloud OpenStack CLI further is to automate everything we’ve done in your favourite scripting language.