Phone Numbers in Rails with Normalizes and Generated Columns

Published in

Nerd For Tech

11 min readFeb 7, 2024

If you’re building an application that uses Vonage Communications APIs, at some point you’re probably going to have to work with phone numbers in some way. Whether your app is making automated text-to-speech phone calls via the Voice API, sending SMS or WhatsApp messages using the Messages API, or implementing two-factor authentication with the Verify API, you’ll need to specify a destination phone number for these interactions.

International Phone Numbers

As you would expect, the API endpoints require phone numbers to be in a specific and consistent format. The number format used by Vonage APIs follows the E.164 international standard, and expects:

An international country dialling code (e.g. 1 for the USA or 44 for the UK)
For the leading + or international access code (e.g. 00) to be omitted
For the trunk access code (e.g. 0) to be omitted
For the number itself to exclude any special characters (e.g. () or -)

For example, a US number would have the format 14155550101, and a UK number would have the format 447700900123.

A typical scenario when building web applications is for the source of your phone number data to be user input captured as part of a sign-up workflow. Ensuring that the input data is in the correct format for later use with the external API can bring some challenges. If you’re building a Ruby on Rails application though, a couple of recently added features can really help in dealing with those challenges.

Example Workflow

Before we dive in to what those features are, let’s think a bit more about that sign-up workflow. In a Ruby on Rails application, we’ll probably define a User model and a UsersController, along with the necessary view templates. Our users/new template, when rendered, might look something like this:

Note that in this example Phone Country Code and Phone Number are separate form fields, and so in a standard Rails implementation will be persisted to the database as separate attributes of our model. For example, we might expect our UsersController to contain code along the following lines:

class UsersController < ApplicationController
  def create
    @user = User.new(user_params)
# rest of create action code
  end
  private
  def user_params
    params.require(:user).permit(:first_name, :last_name, :email, :phone_country_code, :phone)
  end
end

We could combine both pieces of data in a single input in our form, but that would put more responsibility on the user to input the data correctly and potentially make it more difficult to clean up the data on the back-end. Having Phone Country Code as its own form field lets us constrain the data to a specific set of values, for example with a select menu. There are a number of ways that we could implement this field, which I won’t detail, here, but be sure to check out the countries gem and country_select gem.

In a Rails application, we’ll likely be using Rails form helpers in our view templates, and in the case of the Phone Number field itself this will probably be the telephone_field helper (or its alias, phone_field). In the rendered view, this helper creates an HTML <input> element with a type attribute with a value of tel. Other than the value for type (which can be used by mobile phone browsers to determine that a numeric keyboard should be displayed), this field is functionally equivalent to a text type field (in fact, in browsers that don't support tel, this will fall back to a standard text type input).

Unlike some other input types (e.g. email or url), browsers don't support any kind of automatic input validation on this field type. We can of course provide formatting hints to the user alongside the input field, and could perhaps use the pattern attribute on the element in order to constrain the input or use some sort of JavaScript-based solution. As responsible developers though, we shouldn't completely rely on the end-user or on front-end validation -- it's always prudent to clean up our user input before persisting it to a database. In the context of telephone number input, and particularly when supporting a global customer base, this means being prepared for a wide variety of potential number formats being input. Below are just a few examples of different phone number formats used around the world:

77 12 3000
012345-60000
(01234) 500000
06-10000000
0123-400-000
01234/500000-67

This is where Ruby on Rails normalizes method can come in very useful.

Normalizes

The normalizes method was added in Rails 7.1. The method is included in ActiveRecord::Base via the Normalization module, and therefore available to all our Rails models which inherit from the ApplicationRecord abstract class. In use, the method requires an args list of one or more :names (which equate to the attributes that you want to normalize) and a :with keyword argument, the value of which is a Ruby lambda, with the lambda's argument representing the attribute to be normalized.

Below is what usage of normalize might look like if applied to our phone number scenario:

class User < ApplicationRecord
  normalizes :phone, with: -> phone { phone.delete("^0-9").delete_prefix("0") }
end

In the above example, :phone is a string, so within the lambda we are using two methods of Ruby's String class to transform the input value:

The String#delete method. This method returns a copy of the string object it was called on, with all of the characters indicated by the selector argument removed. Here, through use of the caret ^, the selector is indicating that all characters not in the range 0-9 should be removed (i.e. every non-numeric character).
The String#delete_prefix method. Similar to the delete method, this method returns a copy of the string object it was called on, but in this case only removes characters at the start of the string which match the string passed in as an argument. Here it will remove the character '0' (if it is present) from the string returned by the delete method invocation.

With the normalizes invocation added to our User class, we can quickly test out our implementation in the Rails Console, using the normalize_value_for method:

$ rails console
Loading development environment (Rails 7.1.0)
3.1.0 :001 > User.normalize_value_for(:phone, "07900-00-00-00")
=> "7900000000"
3.1.0 :002 > User.normalize_value_for(:phone, "07900 000 000")
=> "7900000000"
3.1.0 :002 > User.normalize_value_for(:phone, "(07900) 000/000")
=> "7900000000"

It was possible to perform this type of data transformation prior to Rails 7.1, and its addition of the normalizes method, but the implementation would have been a little bit more involved.

One option would be to use an Active Record callback such as before_save:

class User < ApplicationRecord
  before_save :sanitize_phone
private
  def sanitize_phone
    phone&.delete("^0-9")&.delete_prefix("0")
  end
end

Another option would be to over-ride the setter method:

class User < ApplicationRecord
  def phone=(value)
    super(value&.delete("^0-9")&.delete_prefix("0"))
  end
end

The normalizes approach though is simpler and cleaner to implement, especially if you need to transform multiple attributes in the same way:

class User < ApplicationRecord
  normalizes :home_phone, :work_phone, with: -> phone { phone.delete("^0-9").delete_prefix("0") }
end

With the other two approaches we’d need to over-ride multiple setters or define multiple helper methods.

Also note the use of Ruby’s safe navigation operator & in those other implementations. This is tells Ruby to skip a method call if the caller is nil, and its use here prevents a NoMethodError being raised in a situation where the :phone attribute was nil. Because of the the way the safe navigation operator works, we need to remember to include it for every method call in the chain. With normalizes, we get this behaviour for free by default. That is to say if :phone is nil, the lambda won't be executed at all.

Caveats

A couple of caveats about our example normalizes implementation:

It doesn’t cover every situation or edge case that you might encounter in production. For example, the delete_prefix("0") part is there because many countries prepend a 0 to their telephone numbers as an access code when dialling internally within the country (known as trunk access), but this 0 is not required when dialling internationally (when instead an international exit code is used in front of the international country code for the country you are dialling to).
Unfortunately, not all countries follow the same approach. First of all, not all countries require a trunk access code. Of those that do most use 0, but there are also a significant number of countries (the USA included) that use 1 as the trunk access code. There are also some other outliers: a few countries use 8, Hungary uses 06, Mexico uses 01, and Mongolia uses 01 or 02.
Some even trickier edge cases are Italy (along with San Marino and Vatican City), which used to use 0 as a trunk access prefix but now includes that 0 as part of the number for both internal and international dialling, and Greece which uses 2 for landlines and 6 for mobile numbers. To deal with all of these edge cases, a more robust approach for our normalizes implementation in production might be to conditionally use different lambdas based on the country code that was entered.
The purpose of normalizes is to ensure that your data conforms to a specific format before persisting it to a database. What it doesn't do is check whether the number is valid and/or trustworthy. To do that, you might want to look at integrating the Vonage Number Insight API as part of your sign-up workflow (although that's beyond the scope of this article).

For the purposes of this article though, let’s assume that we are happy with our data at this point. Now that we have our data though, how do we use it? That brings us to the second Ruby on Rails feature I want to cover: Generated Columns.

Generated Columns

We have our phone number data stored, but :phone_country_code and :phone are separate attributes of our User model. The migration for our User model might look something like this:

class CreateUsers < ActiveRecord::Migration[7.1]
  def change
    create_table :users do |t|
      t.string :first_name
      t.string :last_name
      t.string :email
      t.string :phone_country_code
      t.string :phone
      t.timestamps
    end
  end
end

When creating or updating a User, we set and access those attributes separately:

user = User.new(phone_country_code: "44", phone: "7900000001")
user.phone_country_code # => "44"
user.phone # => "7900000001"

When using that data with Vonage Communications APIs though, for example to Send an SMS using the Messages API, we need to combine the country code and phone number:

message = Vonage::Messaging::Message.sms(message: "Hello from Vonage!")
Vonage.messaging.send(
  to: "447900000001",
  from: "Vonage",
  **message
)

(Note: The above example uses the Vonage Ruby SDK via the Vonage Rails Initializer)

One option would be to combine the data at the point of use:

user = User.find_by(id: 1)
message = Vonage::Messaging::Message.sms(message: "Hello from Vonage!")
Vonage.messaging.send(
  to: "#{user.phone_country_code}#{user.phone}",
  from: "Vonage",
  **message
)

A down-side to this approach is that we could be using this combined data in multiple places in our code-base, and would need to repeat this pattern with each use of the data. If we want to keep our code DRY, a better approach would be to provide a way to access the the already combined data, for example as international_phone_number.

user = User.find_by(id: 1)
message = Vonage::Messaging::Message.sms(message: "Hello from Vonage!")
Vonage.messaging.send(
  to: user.international_phone_number,
  from: "Vonage",
  **message
)

One way of doing this would be to define a method in our User model that combines both values, for example:

class User < ApplicationRecord
  # rest of User code
def international_phone_number
    phone_country_code + phone
  end
end

Rather than defining additional getter methods in our User class though, it might be cleaner if we could somehow just query the data directly from our database.

Prior to Rails 7.0, one way of achieving this would be to define an international_phone_number column in our users table and then use a before_save callback which creates and additional attribute on the object, and calculates its value, before persisting it to the database:

class User < ApplicationRecord
  before_save :set_international_phone_number
private
  def set_international_phone_number
    self[:international_phone_number] = phone_country_code + phone
  end
end

We could then query that column whenever necessary:

user = User.create!(phone_country_code: "44", phone: "7900000001")
user.international_phone_number => "447900000001"

Aside from feeling slightly ‘hacky’, there are other downsides to this approach:

It adds boiler-plate code to our User class
We have code in our User class which partly defines international_phone_number, when ideally we'd want to keep this kind of logic in our ActiveRecord migrations.

Since Rails 7.0 there’s a cleaner solution to this problem: generated columns. A generated column is a kind of virtual column whose value is derived from the value of other columns. In contrast to the example above, all the logic for generated columns is contained at the database level.

This concept isn’t new. MySQL has had Virtual Columns since MySQL 5.7, and several other RDBMSes have equivalent functionality. If you’re building a Rails application though, the chances are that you’re going to be using PostgreSQL at the data persistence layer. PostgreSQL added Generated Columns in PostgreSQL 12.0, and the Rails 7.0.0 release updated the Rails PostgreSQL adapter to support them.

Let’s look at how we can use Generated Columns as part of our implementation.

To add an international_phone_number generated column to our existing users table, we'll need to create a migration to define it:

class AddInternationalPhoneNumberToUsers < ActiveRecord::Migration[7.1]
  def change
    add_column :users, :international_phone_number, :virtual, type: :string,
      as: "phone_country_code || phone", stored: true
  end
end

The first three positional arguments to the add_column method invocation are the standard required arguments for that particular method in a Rails migration. We have a table_name of :users (which is the table that we want to update), a column_name of :international_phone_number, and a type of :virtual. This is then followed by an options hash with three keys which are specific to defining the :virtual column:

:type. This is the data type for the :virtual column, in this case a :string
:as. This specifies how the value for the column should be generated. In this case we're concatenating the phone_country_code and phone fields using PostgreSQL's || operator.
:stored. This is a boolean value which determines whether the value for the :virtual column should be computed when data is written to the columns used to generate it (true), or when the virtual column itself is queried (false).

One caveat to using generated columns is that the columns from which the generated column’s value is derived must be defined on the same table as the generated column itself.

Conclusion

As we’ve seen, there can be challenges associated with using phone number data in our applications, especially when specific international formats are required for that data. As Ruby on Rails developers though, there are a many useful features provided to us, such as the normalizes method and Generated Columns, which can make our code cleaner and our lives easier.

Maybe you’ll use one of these features in your next Rails app to develop something awesome with Vonage Communications APIs. If so, you can sign up for a free Vonage Developer account to get building straight away. If you have any questions or just want to chat, feel free to join us on Twitter or on the Vonage Developer Slack. Happy coding, and see you next time!

Originally published at https://developer.vonage.com/en/blog/phone-numbers-in-rails-with-normalizes-and-generated-columns