Handling Fraudulent In-App Purchases on iOS

The Audiogalaxy mobile apps for iOS and Android have some in-app purchases which allow users to buy add-ons to augment the free service. I've written previously about some interesting observations about implementing each. Continuing that series, here are some more interesting things I've seen when dealing with in-app purchases on iOS, specifically related to fraudulent purchases.

Apple's in-app purchases programming guide describes the steps needed to verify store receipts once you're notified of a successful transaction in the SKRequestDelegate callback. It's tempting to just unlock the feature when you get notified of a successful transaction being complete, but you really should do the extra work required to send over the receipt to Apple for verification, or you risk getting defrauded pretty easily. Ideally, you want to send the receipt up to your own server first (over SSL, obviously) and then verify the receipt using Apple's iTunes receipt verification service.

Since releasing the in-app purchase for iOS, I've seen several fraudulent purchase attempts. For the first version, I took a conservative approach - when notified by StoreKit that a transaction was successfully completed, I unlocked the feature and then sent the receipt up to my server for verification. Additionally, I recorded verification responses from Apple's service but didn't make the device reject the add-on if it failed. The reason I did this was a combination of naivete and being unsure of how things would really work in the wild. What if my server could not be reached? What if Apple's service was down? I didn't want the user to be charged money and be left hanging because of connectivity problems. (Not knowing better, I also assumed that the 'purchase complete' callback was pretty bullet-proof and meant that the user was charged, and that verification was just added formality).

Well, it turns out that StoreKit on jailbroken iOS devices can be easily fooled into believing that a local transaction completed successfully. So with my conservative approach, users with jailbroken phones could easily get in-app features unlocked without paying for them. Thankfully, I caught this before it did serious damage and pushed an app update. Because of the dumb-client architecture of my system, I was also able to easily revoke the purchases made fraudulently (need more reasons to have a server be in control?) What I had originally failed to account for is that the system is built to handle verification failures - if you legitimately fail to verify a transaction because of connectivity problems on either end, just don't mark the transaction as 'finished' and it will automatically get restored the next time StoreKit's queue is initialized (which is why it's also a good idea to write code to handle restored transactions, even if your in-app purchases are only of the consumable variety). This might mean that an add-on won't be available to a user for a while if they paid and you failed to verify the transaction because of networking problems, but they won't be re-charged for it and it will eventually be unlocked correctly.

In case you're interested, here are some examples of fake base64 encoded receipts:

Y29tLnVydXMuaWFwLjcxNDA4NTE0
Y29tLnVydXMuaWFwLjMwODEwMjk5
Y29tLnVydXMuaWFwLjk2NTQyNjQ3

The response from Apple, when these types of receipts are sent for verification, looks like:

{"status":21002, "exception":"java.lang.ClassCastException"}

FWIW, Apple's documentation states that any non-0 response is considered a failure for purchases other than auto-renewing subscriptions. For those, they have a set of well-defined error codes. In a future post, I'll try to cover my experiences implementing auto-renewing subscriptions (and the non-technical, app approval overhead that goes with it).

If you're curious, here's the distribution of where fraudulent purchase attempts came from:

United States - 38.46%
Israel - 14.11%
Canada - 5.45%
United Kingdom - 4.49%
Saudi Arabia - 3.81%
Netherlands - 3.53%
Germany - 1.60%
Mexico - 1.60%
United Arab Emirates - 1.28%
Japan - 1.28%
Others - rest

Don't Let Apple's Latency Mess With Your iOS App

Last week, I released an update for the iOS Audiogalaxy app which featured its first in-app purchase that allows users to 'pin' songs to their devices. Users can try the feature by pinning a limited number of songs for free, and are then prompted to purchase if they wished to continue pinning more songs. As suggested in Apple's SDK documentation, I built and tested the in-app purchase against the sandbox. After everything checked out as I expected, I submitted both the app update and the in-app purchase for approval by Apple.

The application and the in-app purchase were concurrently approved. After verifying that the status of in-app purchase product in iTunes Connect was "Ready for Sale", I released the updated application. It usually takes about an hour for app updates to start propagating to users, so I anxiously waited to get the update on my iPhone and verify that in-app purchasing worked fine. Once I got the update, I tried to purchase the in-app product but it failed. I had written code to handle potential errors, and looking at the error message I realized that the StoreKit API wasn't able to identify the in-app product I was requesting.

I went back and verified that the product identifier was correct, and everything still worked fine in the sandbox. Users had started hitting the same problem - not good! It's definitely not the best first impression for users to upgrade, try new features and immediately see an error. I scoured the web for clues and found a StackOverflow question which suggested that it can take up to 24-36 hours for in-app products to 'go live'. Uh-oh!

There is no mention of such a delay anywhere in Apple's documentation, and neither is it reflected in the status of the in-app purchase product. If I had known, I obviously would have waited to release the update. Apple doesn't provide the ability to roll back to previous versions, so basically I was SOL. If indeed it took ~24 hours for the in-app product to show up, it could be disastrous. The fallout could include:

• Disappointed users. There is nothing more disappointing that looking at the "What's New" section of an app before upgrading, getting excited and then watching the feature just not work as expected.
• Negative reviews/ratings. App Store ratings/reviews provide a medium for users to vent their frustrations or to shower praise. For developers, it's really a matter of making sure that the people with exceedingly pleasurable experiences with your application exceed the number of people with even remotely dissatisfying experiences. Audiogalaxy was a 5-star rated app, but this snafu could potentially affect that negatively, really fast.
• Personal disappointment. I take a lot of pride in making sure that I deliver delightful experiences to users, and this would be the exact opposite of delightful. The hurt would be tempered by the fact that this was beyond my control, but it would hurt nonetheless.
• Lost revenue. From my experience with in-app purchases on Android (and now also on iOS) it was abundantly clear to me that purchase volume is very high in the first few days of a new in-app purchase being available. Users upgrade, try what's new and buy it if they like it. If they're unable to, they might never come back to buy it.

I was really hoping that the problem was the delay-to-propagate issue and not anything more serious. Damage would be limited if the in-app purchase went live before too many users upgraded and started using the new feature, and I was secretly hoping that this was a matter of a few hours instead of 24-36 as suggested in the SO question.

I started monitoring how many users were pulling down the update and trying this new feature (yay for building simple monitoring tools that are fully in your control) and also dumping these userIDs into a table so I could keep track - a subset of these users were (hopefully) trying to purchase the add-on, but were failing to do so. The number started growing rapidly, and I began to panic. 5-6 hours later, the in-app product still wasn't live, negative reviews had started showing up and folks had started emailing us to let us know.

Watching user behavior, I knew something needed to be done. We decided that the best option was to make it impossible to even attempt to purchase the add-on. I updated the server to make it believe that all iOS users had already purchased the add-on - the app would then automatically hide all purchase related options. The plan was to just let everyone use the feature for free until the in-app purchase showed up for real. I wrote the code, tested it and pushed it to production. I watched it work for a bit, and then went to sleep.

The next morning, 14 hours after the status was "Ready for Sale", I was finally able to purchase the in-app product from the app. Thankfully, the server tweaks had helped and there were no new negative reviews or support emails. I rolled back the code from the previous night and waited for purchases to start showing up. Within minutes, I could see folks buying so I knew things were working fine now.

The last thing that needed to be done was informing users who might have previously had trouble and might potentially never try buying again. I wrote a script to run through the list of all users who had tried the feature and to send them a personal note thanking them for their support, explaining what had happened and letting them know that the in-app purchase was now available if they wished to purchase it. A surprisingly large percentage of users went ahead and purchased it, and a few also went back and updated their reviews.

I took away three lessons from this experience -

• Design your apps/clients to be as dumb as possible and the server as much in control as possible. We've always tried to do that with Audiogalaxy, and it certainly helped in this situation (as it has on a few previous occasions).
• Spamming all your users isn't a good idea, but it's perfectly OK to email a subset of users directly impacted by specific issues. Users are receptive to personally addressed emails with honest dialogue, and don't consider it spam.
• Wait for 24 hours before releasing iOS app updates if you have added new in-app purchases.

All in all, this didn't turn out as badly for us as it could have - but it shouldn't have happened. I still believe Apple is at fault for not communicating this information to developers, but that's not a rare occurrence for Apple - it's best to anticipate and prepare for the unexpected. I hope this blog post helps other who might be planning to add in-app purchases to their apps. In addition to better documenting such gotchas, it would also be awesome if Apple added the ability for developers to roll back to previous versions (or at least 1 previous version) like Google's Android Market recently did.