Until recently, adding new subscribers to my newsletter was kind of a hack I was not very proud of. Subscribing to my newsletter worked this way:

• People submit the form at the bottom of one of my articles.
• The subscription controller checks if the provided email address is valid and whether a first name was given.
• The controller itself opens a connection to MailChimp's API backend adding the subscriber to my newsletter list, which triggers the confirmation email.

It was easy to implement and I needed a working solution quickly, but there is so much wrong with this approach:

• It is slow
  Interacting with a third party services takes time, so visitors had to stare at a disabled submit button for a few seconds.
• It is error prone
  If something went wrong (an API downtime or a network issue) I would indeed communicate that with a clear error message, but the potential subscriber was lost - no-one comes back to try again if it does not work the first time.
• It is tightly coupled
  If the code for interacting with MailChimp's API resides in the controller, swapping the service provider requires changing controller code and tests.
• There is no backup
  Even though MailChimp makes exporting your list's subscribers easy, I had no local copy of my subscribers (so checking for already subscribed visitors takes just as long as adding a new subscriber).

This had to be reworked.

Switching to an ActiveRecord model

The first thing I did was refactoring the code, making "subscription" an ActiveRecord model (prior to that it was just a Ruby class with ActiveRecord's validations mixed in). Moving the interaction with MailChimp's API to an after_save-hook made the controller code clean again and I got a local copy of my subscribers list for free.

However, it was still slow and tightly coupled. Introducing a service object would have removed the tight coupling and made testing easier (while still being slow) but I decided to completely remove the interaction with the third party API from my application.

When you think about it, there is no need for the visitor to wait for a response from MailChimp's API: I have his email address and his first name, which is all I need. Storing this information in my application's database happens instantly. What happens afterwards really is my job and should not affect the visitor's experience with my website.

So: Background jobs!

I took a look at Sidekiq which is a really neat solution for implementing background jobs in your Rails application. Just tell a worker what to do, throw it into a queue and forget about it.

The downside is that you have to have a Redis server running which has to be monitored (in case it dies and needs to be restarted). This is one more moving part in the already complex Rails stack and I decided that it was overkill for my needs.

cron to the rescue

All I needed was a way to mark new subscribers as "unhandled" (when their email address needs to be sent to MailChimp's API) and later "opted_in" (when a subscriber actually joins the newsletter list by clicking the link in the confirmation email) in my application's database.

A small Ruby script would then periodically execute two tasks:

• Check the local database for unhandled subscribers and send their email addresses to MailChimp's API (if something goes wrong here, the script will just try again later, marking only those subscribers as "handled" which MailChimp reports back).
• Compare the list of all subscribers in MailChimp to the subscribers not yet marked as "opted_in" locally (thus finding out which subscribers clicked the confirmation link since the last check) and mark them as "opted in" (a more elegant solution would have been using a web hook, but then again - overkill for my needs).

This is the script I came up with in its first iteration:

#! /usr/bin/env ruby

require 'active_record'
require 'yaml'
require 'mailchimp'

# ==============================================================================
# Basic subscription class
# ==============================================================================

class Subscription < ActiveRecord::Base
  scope :unhandled, -> { where(handled: false) }
  scope :not_yet_opted_in, -> { where(handled: true, opted_in: false) }
end

# ==============================================================================
# Configuration
# ==============================================================================

environment = ENV['RAILS_ENV'] || 'development'
db_config = YAML::load(File.open(File.join(Dir.pwd, 'config', 'database.yml')))[environment]
mailchimp = Mailchimp::API.new('00000000000000000000000000000000-us8')
list_id = '0000000000'

ActiveRecord::Base.establish_connection(db_config)

# ==============================================================================
# Handle new subscriptions
# ==============================================================================

if Subscription.unhandled.any?
  unhandled_subscriptions = Subscription.unhandled.map do |subscription|
    {
      :email      => { 'email' => subscription.email_address },
      :merge_vars => { 'fname' => subscription.first_name },
      :email_type => 'html'
    }
  end

  # Send the list to MailChimp and get an array of added email addresses back
  new_email_addresses = mailchimp.lists.batch_subscribe(list_id, unhandled_subscriptions, true, false, true)['adds'].collect { |subscription| subscription['email'] }

  # Mark all subscriptions as handled whose email addresses were added
  Subscription.where(email_address: new_email_addresses).update_all(handled: true)
end

# ==============================================================================
# Handle new Opt-Ins
# ==============================================================================

# Get an array of all email addresses that MailChimp has for this list
opted_in = mailchimp.lists.members(list_id)['data'].select { |member| member['status'] == 'subscribed' }.map { |member| member['email'] }

# Mark all subscriptions whose email addresses are in MailChimps list (and thus
# have opted in) as opted in
Subscription.not_yet_opted_in.where(email_address: opted_in).update_all(opted_in: true)

ActiveRecord::Base.connection.close

All that's left to do is configure cron to run this script every 5 minutes. My applications all run under the app user, so I opened its crontab (crontab -e) and added the following line:

*/5 * * * * /bin/bash -lc /home/app/relativkreativ.at/codebase/subscription.rb

Note that a cron job runs with basically no environment. Since I use rbenv in production, I have to make sure that the rbenv environment is loaded (by starting a non-interactive login-shell with bash -lc) and all required variables are set (my RAILS_ENV is set system-wide via export RAILS_ENV=production in the file /etc/profile.d/rails_env.sh) before calling the script.

Now, at the latest 5 minutes after a visitor subscribes to my newsletter, the new email address is added to my newsletter-list in MailChimp which in turn sends the opt-in confirmation email. The first time the script runs after the new subscriber has clicked the confirmation link, my database is updated and I know I have a new adequate subscriber.

What's missing in this solution is taking care of people who unsubscribe. Of course these need to be removed from the local database.

The conclusion

Even though there are loads of excellent gems out there, you do not need to resort to one for every simple task. Think about your requirements and use the right tool for the job™.