7

We have a (mostly) successful implementation of push notifications to iOS and Android devices through Azure Notification Hubs.

The problem is that some of the iOS devices are apparently never receiving notifications that are sent by Azure Notification Hubs.

We use templates and tags to direct the messages to the appropriate devices. The tags are interest topics, and never user-specific, so we're expecting one notification for a tag to be pushed to all devices subscribed to that tag.

The Android devices seem to receive their notifications flawlessly, but the iOS devices are not consistent. Most of them work. A couple do not.

We are well aware that push notifications are delivered with best effort and have no guarantee of reliability, but our limited testing has revealed more devices which consistently fail to receive push notifications than seems unreasonable (more than two failures from about a dozen devices).

Here's the setup:

We have a simple C# routine in the back end which connects to Azure Notification Hubs and sends notifications to Azure:

var outcome = await hub.SendTemplateNotificationAsync(properties, tag);

We have used the GetAllRegistrationsAsync method to make sure that every device we are checking has successfully registered and is using the correct template. Every device is registered, all the templates are correct.

We are not in "test mode"; the enableTestSend parameter of NotificationHubClient.CreateClientFromConnectionString is set to False.

Troubleshooting:

When we send the notification out, most devices receive the notification and, in the specific case we're testing, update the badge counter with the correct number.

However, a couple of devices do not seem to get the notification. One of the devices did get the notification after we rebooted the device, but after that it stopped.

Using the above mentioned GetAllRegistrationsAsync method, we have verified that the problem devices are correctly registered on Azure and have the correct tags and templates.

We were able to determine the device tokens of the problem devices from the Azure registrations. We used a PHP script which communicates directly with APNS to send a notification just to the problem devices using their device tokens. Every time, the device receives this direct-send notification. It's only the notifications from Azure which are unreliable.

When we examine the Azure Notification Hub Monitor page, we see these metrics for the past 24 hours:

  • 967 APNS Successful Notifications
  • 3 APNS Bad Channel Errors
  • 2 APNS Expired Channel Errors
  • 4 APNS Errors

... and no other errors reported for APNS or for Azure in general. The failure rate we're seeing should have produced an error count over 20.

We have not been able to determine which device tokens were responsible for the errors; is there a way to get this information from Azure?

We're at a loss to explain why we can send notifications directly to these devices over APNS itself, but not through Azure, and why it is that Azure doesn't report more errors than it does.

Any suggestions or insights?

1 Answer 1

7

It's quite possible that you have some sandbox device tokens in your database (I'm not sure if the device tokens are stored in your server or in Azure Notification Hub). When trying to send a notification with a sandbox device token to the production push environment, an InvalidToken error is returned by Apple, and the connection is closed.

Very often, by the time the server that sends push notifications to Apple's APN server gets the error response, it has already sent many more notifications (possibly with valid tokens), and all of which are discarded by Apple. At this point, new notifications are accepted by Apple only after a new connection with APNS is established, so messages that were sent after the invalid token to the old connection need to be resent. It is possible Azure don't handle this resending correctly.

As you said, the Azure Notification Hub Monitor page shows a few errors. I suspect that 3 APNS Bad Channel Errors means invalid device tokens. I don't know how many invalid device tokens you actually have in the DB, but even one can cause many notifications with valid tokens not to be accepted by Apple.

The best solution is to test all the device tokens in the DB and figure out the ones that are invalid and delete them.

4
  • That seems like a fairly reasonable explanation, but wouldn't your explanation mean that we should not be able to send the direct message through APNS, using the exact same device token as Azure uses? Or might Apple be selectively rejecting incoming messages based on the sender IP (or something like that)? Alternately, do you know of a way to get those invalid device tokens out of Azure? (We're not doing the registrations; the devices communicate directly with Azure.)
    – Alan McBee
    Commented Jul 3, 2014 at 1:42
  • @AlanMcBee The device tokens you tried to send directly are probably valid, otherwise they would never work. The problem is probably caused by other device tokens you are not aware of. I don't know how you can remove invalid tokens from Azure. I've never worked with Azure. Did you register devices with Azure while testing development build (with sandbox push environment)? Those are the most likely problematic device tokens.
    – Eran
    Commented Jul 3, 2014 at 1:52
  • 1
    We probably should have un-registered all of the pre-production registrations that we created while testing, so that there wouldn't be any problems. But all of the problem devices have successfully registered their tokens with Azure after they have had the production app installed, with the production certificate. I expected individual invalid device tokens to be eventually removed from registration. We actually have a support ticket open with Microsoft to help track down individual device errors. I'll keep this thread posted.
    – Alan McBee
    Commented Jul 4, 2014 at 19:01
  • 1
    Postscript. The issue was indeed invalid device tokens. There is not (or was not at the time) an efficient way to detect those sandbox tokens and remove them; we wound up having to unregister all the tokens and start over. Moving forward, we would make sure to never use the same service bus + notification hub for pre-prod (sandbox) registrations as we do for prod. Microsoft PMs are/were considering ways to ameliorate, but I don't know what came out of that.
    – Alan McBee
    Commented Dec 30, 2014 at 18:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.