Developer Guide to Failed Recurring Payment Retry System

WooCommerce Subscriptions is a premium plugin, and version 2.1 introduced a new system to automatically retry a recurring payment that previously failed.

This guide provides a technical overview of the Failed Recurring Payment Retry system and is intended for developers looking to customize or otherwise interact with the retry system.

We recommend reading the Store Owner Guide to the Failed Payment Retry System for a non-technical introduction to the retry system.

Retry System Components

The retry system is made up of a number of different components, each of which implements a distinct aspect of the retry system. These components are:

• WCS_Retry_Manager: Manages the entire retry system, from loading all components to hooking into the normal failed payment flow, checking retry rules and applying a rule to retry a failed renewal payment when required.
• WCS_Retry_Rules: Sets up the default store-wide rules for retrying failed automatic renewal payments and provides methods for working with rules, like get_rule().
• WCS_Retry_Rule: Represents instance of a retry rule and provides methods for retrieving and checking a rule’s properties. Used by WCS_Retry_Rules->get_rule() and WCS_Retry->get_rule().
• WCS_Retry: Represents instance of a retry and provides methods for retrieving and checking properties on a retry, like get_order_id() and get_rule().
• WCS_Retry_Store: Provides an extensible interface to store a retry in the database.
• WCS_Retry_Post_Store: Implements WCS_Retry_Store to store retry details in the WordPress posts table as a custom post type.
• WCS_Retry_Database_Store: Implements WCS_Retry_Store to store retry details in the wcs_payment_retries custom table.
• WCS_Retry_Hybrid_Store: The hybrid store acts as a bridge between the two default stores and migrates retries from the post store to the database store when a retry is retrieved from the post store. This store also extends WCS_Retry_Store.
• WCS_Retry_Background_Migrator: This class extends the WCS_Background_Upgrader class and handles the migration of retries using WCS_Retry_Migrator in the background via an Action Scheduler action.
• WCS_Retry_Migrator: The retry migrator contains the logic behind the migration from the post store to the database store.
• WCS_Retry_Stores: Managers the two default store interfaces and contains functions to access them. Use WCS_Retry_Stores::get_post_store() or WCS_Retry_Stores::get_database_store() to get the stores.
• WCS_Retry_Email: Manages emails sent as part of the retry process by registering the custom retry email classes and sending emails on relevant hooks.
• WCS_Email_Payment_Retry: Controls the email template sent to store owners when an attempt to automatically process a subscription renewal payment has failed and a retry rule has been applied to retry payment in the future.
• WCS_Email_Customer_Payment_Retry: Controls the email template sent to the customer/subscriber when an attempt to automatically process a recurring payment has failed and a retry rule has been applied to retry payment in the future.
• WCS_Retry_Admin: Sets up the administration UI elements, including the Automatic Failed Payment Retries meta box and the setting to Enable the Retry System.
• WCS_Retry_Table_Maker: Extends WCS_Table_Maker, and defines the version and table definition for the retries custom database.

Retry Process Flow

The Store Owner Guide provides a non-technical overview of the Retry Process. This section provides a technical guide, including details of hooks involved in the process.

How general retry process proceeds:

1. The 'woocommerce_subscription_renewal_payment_failed' action is triggered in WC_Subscription->payment_failed() after a renewal payment fails.
2. The WCS_Renewal_Retry_Manager::maybe_apply_retry_rule() is called, as it’s attached to 'woocommerce_subscription_renewal_payment_failed'.
3. WCS_Renewal_Retry_Manager::maybe_apply_retry_rule() checks:
   • the subscription is manual: $subscription->is_manual().
   • automatic retry is possible with the payment method on the subscription: $subscription->payment_method_supports( 'gateway_scheduled_payments' ).
   • the last order is a renewal: wcs_order_contains_renewal( $last_order ).
   • there is a retry rule for this stage of the retry process and for this specific order: WCS_Renewal_Retry_Manager::rules()->has_rule( WCS_Renewal_Retry_Manager::store()->get_retry_count_for_order( $renewal_order->id ), $renewal_order->id ).
4. If all these conditions pass:
   • A new pending retry is saved to correspond to the rule: WCS_Renewal_Retry_Manager::store()->save( $retry ).
   • Status of the renewal order is updated according to the rule: $order->update_status( $new_status ).
   • Status of the subscription is updated according to the rule: $subscription->update_status( $new_status ).
   • Interval time defined by the retry rule is then used to set the retry date/time on the subscription: $subscription->update_dates( array( 'payment_retry' => gmdate( 'Y-m-d H:i:s', gmdate( 'U' ) + $retry_rule->get_retry_interval( $retry_count ) ) ) ).
5. When the scheduled time for the retry event arrives, the 'woocommerce_scheduled_subscription_payment_retry' action will be triggered passing callbacks the ID of the renewal order to which the retry relates.
6. The WCS_Renewal_Retry_Manager::maybe_retry_payment() is called, as it is hooked to 'woocommerce_scheduled_subscription_payment_retry'.
7. WCS_Renewal_Retry_Manager::maybe_retry_payment() checks:
   • Retry still has pending status
   • Last order still needs payment: $last_order->needs_payment()
8. If these checks fail, the status of the retry is transitioned to 'cancelled'.
9. If they pass, WCS_Renewal_Retry_Manager::maybe_retry_payment() will transition the retry to the 'processing' status and then check:
   • Last order still has the status defined by the retry rule applied to it (if a status was defined): $last_order->has_status( $last_retry->get_rule()->get_status_to_apply( 'order' ) ).
   • Subscription still has the status defined by the retry rule applied to it (if a status was defined): $subscription->has_status( $last_retry->get_rule()->get_status_to_apply( 'subscription' ) ).
10. If these additional checks fail, the status of the retry will be transitioned to 'cancelled'
11. If they pass, WCS_Renewal_Retry_Manager::maybe_retry_payment() will then:
    • Update the subscription status to on-hold in preparation for payment (Subscriptions uses this status immediately before payment regardless of status defined by the retry rule to avoid compatibility issues with payment gateways that expect the subscription to have on-hold status, as it would for normal recurring payments): $subscription->update_status( 'on-hold' ).
    • Tell the payment gateway on the subscription to process payment for the last order: WC_Subscriptions_Payment_Gateways::gateway_scheduled_subscription_payment( $subscription ).
12. WCS_Renewal_Retry_Manager::maybe_retry_payment() will then check the order again and if it does not need payment, the status of retry is transitioned to 'complete' status as the last payment must have been processed correctly.
13. If the last order still needs payment, the retry is transitioned to 'failed' status as the last payment did not process correctly.
14. If payment failed, WC_Subscription->payment_failed() will trigger 'woocommerce_subscription_renewal_payment_failed' again and the process repeats from step 1.

Customizing the Retry Process

The retry process is based on a set of retry rules, as explained in the Store Owner Guide to the Failed Payment Retry System.

These rules can be customized by changing both the default rule applied to all failed payments in the store and/or one specific rule applied to a given order.

Rule Data Structure

Retry rule data can take two forms:

• A raw retry rule is an array. This is the form used to store the default rule set in the protected WCS_Retry_Rules->default_retry_rules property and to store the rule applied for a specific retry in the database.
• An instance of WCS_Retry_Rule (or child class of WCS_Retry_Rule). This is the form used to work with a specific rule.

Raw Rule Data Structure

The raw rule format is an array with the following values:

• retry_after_interval: Amount of time, in seconds, between when the rule is applied and when to reattempt payment. Note: This interval accumulates between rules. For example, consider a store with two rules, the first with an interval of 24 hours and the second with an interval of 48 hours. If the 1st retry attempt fails, the 2nd retry attempt will be run 48 hours after it fails, not 48 hours after the first payment failed. This is 72 hours after first payment failure.
• email_template_customer: Class name of the email template to send the customer when the retry rule is applied (i.e. the payment fails). If empty, no email is sent.
• email_template_admin: Class name of the email template to send the store owner when the retry rule is applied (i.e. the payment fails). If empty, no email is sent.
• status_to_apply_to_order: Status to apply to the renewal order between when payment fails and when it is attempted again. This is not the status applied after the payment attempt for this retry rule fails. It is the status applied at the time the retry rule is applied, which happens when the previous payment attempt failed. This can be either the full, internal status name, e.g. wc-pending or shorthand status name, e.g. pending.
• status_to_apply_to_subscription: Status to apply to the subscription between when payment fails and when it is attempted again. This is not the status applied after the payment attempt for this retry rule fails. It is the status applied at the time the retry rule is applied, which happens when the previous payment attempt failed. This can be either the full, internal status name, e.g. wc-on-hold or shorthand status name, e.g. on-hold.

This snippet provides an example of rule in the raw, array data structure.

array(
    'retry_after_interval'            => DAY_IN_SECONDS,
    'email_template_customer'         => 'WCS_Email_Customer_Payment_Retry',
    'email_template_admin'            => 'WCS_Email_Payment_Retry',
    'status_to_apply_to_order'        => 'pending',
    'status_to_apply_to_subscription' => 'on-hold',
),

Rule Class Data Structure

The WCS_Retry_Rule class is used by default to instantiate retry rule data into the object used in WCS_Retry_Manager and elsewhere.

The class used to instantiate rule data can also be customized with the 'wcs_retry_rule_class' filter. In most cases, this is unnecessary. It is only necessary to achieve behavior not customizable via custom rule filters detailed below.

This snippet provides an example of using a custom rule class.

function eg_my_custom_retry_rule_class( $default_retry_class ) {
    return 'EG_Retry_Rule';
}
add_filter( 'wcs_retry_rule_class', 'eg_my_custom_retry_rule_class' );

Custom Storewide Rules

The retry system uses a default set of rules for managing all failed payments in the store. These rules are defined in WCS_Retry_Rules::__construct() and accessed via WCS_Retry_Rules::get_rule().

Customizing the default retry rules makes it possible to apply a new set of rules that are better suited to unique requirements for your store. For example, if a store only sells annual subscriptions, you may wish to use retry rules that continue retrying for up to 30 days after the initial payment failed, rather than the much shorter default.

The 'wcs_default_retry_rules' filter makes it possible to customize the default rules. This filter passes callbacks an array of rules in the raw array rule data structure, and expects to receive the same from callbacks.

Note: To apply code to the 'wcs_default_retry_rules' filter, your add_filter() call will need to occur before WCS_Retry_Rules is first instantiated, which is attached to the 'woocommerce_subscription_renewal_payment_failed' and 'woocommerce_scheduled_subscription_payment_retry' hooks.

Example Custom Default Rules

This snippet provides a complete set of custom rules that will:

• Retry a payment 5 times over the course of a month, instead of the default 7 days
• Implement more advanced dunning than the default rules, by sending three different emails to the customer
• Only notify the store owner of the failed payment via email on the first and last retry attempt
• Leave the subscription active for the first 7 days to provide a grace period before blocking access to virtual content linked to the subscription

function eg_my_custom_retry_rules( $default_retry_rules_array ) {
    return array(
            array(
                'retry_after_interval'            => 3 * DAY_IN_SECONDS,
                'email_template_customer'         => '',
                'email_template_admin'            => 'WCS_Email_Payment_Retry',
                'status_to_apply_to_order'        => 'pending',
                'status_to_apply_to_subscription' => 'active',
            ),
            array(
                'retry_after_interval'            => 4 * DAY_IN_SECONDS,
                'email_template_customer'         => 'EG_Email_Customer_Payment_Retry_First_Nag', // custom email
                'email_template_admin'            => '',
                'status_to_apply_to_order'        => 'pending',
                'status_to_apply_to_subscription' => 'active',
            ),
            array(
                'retry_after_interval'            => WEEK_IN_SECONDS,
                'email_template_customer'         => '', // avoid spamming the customer by not sending them an email this time either
                'email_template_admin'            => '',
                'status_to_apply_to_order'        => 'pending',
                'status_to_apply_to_subscription' => 'on-hold',
            ),
            array(
                'retry_after_interval'            => WEEK_IN_SECONDS,
                'email_template_customer'         => 'EG_Email_Customer_Payment_Retry_Second_Nag', // custom email
                'email_template_admin'            => '',
                'status_to_apply_to_order'        => 'pending',
                'status_to_apply_to_subscription' => 'on-hold',
            ),
            array(
                'retry_after_interval'            => WEEK_IN_SECONDS,
                'email_template_customer'         => 'EG_Email_Customer_Payment_Retry_Final_Nag', // custom email
                'email_template_admin'            => 'WCS_Email_Payment_Retry',
                'status_to_apply_to_order'        => 'pending',
                'status_to_apply_to_subscription' => 'on-hold',
            ),
        );
}
add_filter( 'wcs_default_retry_rules', 'eg_my_custom_retry_rules' );

Custom Individual Rule

You may wish to apply different retry rules for different products, billing schedules, payment amounts or other conditions. To do this, you can customize a specific retry rule based on the order ID and its position in the retry queue.

An individual rule can be customized in two ways, depending on the data structure of the rule.

To customize the raw rule in array format, use the 'wcs_get_retry_rule_raw' filter. To customize the instantiated rule object, use the 'wcs_get_retry_rule' filter.

Callbacks on both of these filters receive 3 parameters:

1. $rule: this will be:
   • an array representing the rule with the array keys described above for 'wcs_get_retry_rule_raw'.
   • an instance of WCS_Retry_Rule for 'wcs_get_retry_rule'.
   • null if there is no rule for this $retry_number and $order_id.
2. $retry_number: the position in the retry queue, starting at 0. For example, after the first payment failure, there have been no retries, so the $retry_number would be 0. If the retry fails after applying this first rule, to get the rule for the 2nd retry, the $retry_number would then be 1.
3. $order_id: the ID of the order for which this rule relates.

Example Custom Individual Raw Rule

This snippet changes the email template sent to customers for a product with ID 30.

function eg_my_custom_retry_rule( $rule_raw, $retry_number, $order_id ) {

    $order       = wc_get_order( $order_id );
    $has_product = false;

    foreach ( $order->get_items() as $line_item ) {
        if ( $line_item['product_id'] == 30 ) {
            $has_product = true;
            break;
        }
    }

    if ( $has_product && ! empty( $rule_raw['email_template_customer'] ) ) {
        $rule_raw['email_template_customer'] = 'EG_Email_Customer_Payment_Retry_Product_Thirty';
    }

    return $rule_raw;
}
add_filter( 'wcs_get_retry_rule_raw', 'eg_my_custom_retry_rule' );

Example Custom Individual Rule Object

This snippet uses a custom retry rule class and interval for annual subscriptions.

function eg_my_custom_retry_rule( $rule, $retry_number, $order_id ) {

    $subscription = wcs_get_subscriptions_for_order( $order, array( 'order_type' => 'renewal' ) );

    if ( ! empty( $subscription ) && 'year' === $subscription->billing_period ) {

        $existing_rule_raw = $rule->get_raw_data();

        if ( ! empty( $existing_rule_raw['retry_after_interval'] ) ) {
            $existing_rule_raw['retry_after_interval'] = WEEK_IN_SECONDS;
            $rule = new EG_Retry_Rule( $rule->get_rule_raw() );
        }
    }

    return $rule;
}
add_filter( 'wcs_get_retry_rule', 'eg_my_custom_retry_rule' );

Testing the Retry System

After creating custom retry rules, or to check whether your gateway is compatible with the retry system, you can test the retry system with the following process.

Step 1: Enable the Retry System

Before testing, ensure that you have enabled the Retry System in your store.

Step 2: Trigger a Failed Recurring Payment

Depending on your gateway, the way to trigger a recurring payment failure will differ. For Stripe and other payment gateways that support admin payment method changes you can:

1. Purchase a subscription using a payment method which supports payment date changes
2. Go to: WooCommerce > Edit Subscription for that subscription
3. Click the pencil icon next to Billing Details
4. Enter dummy payment token meta data so that the payment will fail
5. Click Save Subscription
6. After the page has reloaded, click the Actions select box in the Subscription Actions metabox
7. Click Process Renewal
8. Click Save Subscription

This creates a renewal order, records the failed payment on that renewal and applies the first retry rule to that order.

Step 3: Monitor Retries

At this stage, you can view the details of the Pending retry via the interfaces detailed in the Store Manager guide to Monitoring Retries.

If your retry rule worked, you should now be able to see:

• A Renewal Payment Retry date on the WooCommerce > Edit Subscription administration screen
• A Pending retry in the Automatic Failed Payment Retries metabox

Step 4: Trigger the Retry (Optional)

If you do not want to wait until the retry is triggered automatically, you can trigger the retry immediately.

To trigger a scheduled payment retry hook immediately:

1. Make sure your store is running in debug mode by setting the WP_DEBUG constant
2. Visit your WordPress administration dashboard
3. Go to: Tools > Scheduled Actions
4. In the search box, enter the ID of your test order
5. Find the row with the hook 'scheduled_subscription_payment_retry' and the status pending
6. Hover over the row and click Run

This immediately triggers the 'scheduled_subscription_payment_retry' hook.

Retry Migration to Custom Table

When the retry system was introduced in 2.1, retries were a custom post type and were stored in the WordPress posts and postmeta tables. To improve the performance of the retry system, in version 2.4 we introduced a custom table to store the retry data. With the introduction of this custom table, we needed to migrate the existing data.

Retries are migrated to the custom table in 2 ways:

1. On the fly: while retries still exist in the posts table, the WCS_Retry_Hybrid_Store is used to act as a bridge between the two data stores. When a retry object is requested (e.g. via WCS_Retry_Manager::store()->get_retry( $retry_id )), the hybrid store will first check if the retry exists in the post table and if it does, it will migrate the retry data to the new custom table before returning the retry object.
2. In the background: when the store upgrades to 2.4, or any version after 2.4, a migration action will be scheduled via the Action Scheduler library. When this action is triggered, the WCS_Retry_Background_Migrator class will migrate retries in batches, rescheduling itself every 60 seconds (by default) until all retries have been migrated. WCS_Retry_Background_Migrator is initialized by and stored on WCS_Retry_Manager as protected variable.

How do I track the progress of the retry data migration?

The status of the retry data migration is displayed in the System Status.

1. Go to WooCommerce > Status.
2. Scroll down to Retries Migration Status.

How can I tell how many retries still need to be migrated?

To see how many retries you still have stored in the posts table:

1. Go to WooCommerce > Status and scroll down to the Post Type Counts section.
2. The number next to payment_retry is the number of retries which haven’t been mirgrated.

If there isn’t any payment_retry row displayed in this table, there aren’t any retries still stored in the posts table.