Tuesday, March 10, 2009

Review Silliness

I wish I had some visibility into the process used for the App Store reviews at Apple. I'm sure those reviewers are in a tough job and all. Probably a lot of pressure, too much work, etc. I know. But, there have just been so many examples of silly rejections lately, I have to do a quick post on it.

Most recent: Loren Bichter, author of the great iPhone Twitter client Tweetie, just had an update to his application rejected because of a hashtag it found in the Twitter timeline.

By this measure, they should be pulling Mobile Safari off of the phone because it lets you see all sort of nasty words and pictures that Ms. Grundy certainly would not want us to see. Sheesh.

NOTE: Loren re-submitted Tweetie and was promptly approved. To their credit, Apple does often move promptly to fix individual problems when they become aware of them. Now if they can just fix the system that allowed the problem in the first place. I'm starting to hear bits and pieces of why this is the way it is.

Chris Stewart, who runs iPhone Dev SDK had his app rejected because it "provided information that is in violation of the iPhone SDK". His app? An RSS reader that pulled feeds from his site's forums, which are about programming using the iPhone SDK - the official SDK. I've hung out there a fair bit, and I can't recall ever seeing anybody promoting anything illegal. In fact, people looking for Jailbreak info are usually referred to other sites because his site is about programming the iPhone SDK, specifically people who want to get apps up on the App Store (isn't that meta?).

Before that, I heard from several developers whose apps were rejected for having "too little content", or for "not providing enough functionality". First of all, there is nothing in the SDK agreement that says you have to have a certain baseline amount of functionality, and second of all, all of these rejected applications did as much or more than hundreds of applications already on the App Store. In at least two cases, the authors re-submitted their apps without any changes, and they were accepted! How can that not be arbitrary? It's not deterministic. If you do the same exact thing twice, you should get the same exact results, unless there's some kind of randomness in the system. Obviously, Apple's review process has some amount of randomness to it or, in other words, is arbitrary.

Making arbitrary decisions, or those that aren't arbitrary, but are perceived that way, is bad for the platform. It's going to sour developers on the process and on Apple. Many iPhone developers (me included) are among the most fervent fans of Apple and their products. We buy a lot of their products, and we recommend them. We also develop the applications that make the platform exciting and constantly improving. In terms of long term strategies, constantly annoying your biggest fans and most stalwart customers is not a great one.

I understand that Apple is trying to maintain a certain level of quality and content on the App Store, and that they have to pay for the infrastructure to distribute the free applications, so they understandably don't want to pay money to deliver useless but free programs to thousands of users. Yes, it's their house, and we've all agreed to play there, but if they keep changing the rules, some people might start picking up their balls and going home.

There's got to be way for Apple to do what they need to do without annoying developers and without making it look like apps that get rejected did so because somebody was in a bad mood.


Mike said...

Apple has been killing me with their review process as well. 2 1/2 weeks for my last update to an app that has been in the App Store since August 2nd. I have had an app 'In Review' since Feb. 16th now. Every day for the last week I have called them trying to get some information but they say nothing. The app is a one screen utility that quite obviously doesn't use any private API calls and is in no way objectionable.

Things are getting ugly with Apple these days...I really hope they fix things.

JFMartin said...

They are over loaded. The whole process is overloaded. They didn't anticipate that obviously. The review criteria is also very subjective (at least, they sometime look subjective and it is the same). Some App in the App Store are so ULGY they should have not maed it by Apple deisgn standards... but they are there...

Andrew said...

Another strategy of the App Store Approval Process is to leave the App in review status forever.

In my case I rejected the App myself after 3 month of waiting, because in the meantime I had several updates for the App.
That was two weeks ago and the Approval Process began again.

Darren Stone said...

After an app rejection of my own, my impression of App Store reviewers is that they're like junior QA testers, sometimes *very* junior, and they need to be handled as such. The big difference of course is that these testers know next-to-nothing about the software they're supposed to test; a difficult job for anybody.

Ultimately, the reviewer/tester must pass or reject an app based solely on his or her own judgement and own understanding of the rules, and it's very easy for them to make a bad judgement based on a misunderstanding. However, in my experience it's just as easy to steer them back to the right track if they've made a mistake. I think it's unrealistic to expect anything more from a reviewer.

In my case, the kid I had to deal with responded promptly and professionally, and it took only a simple clarification to get past the review. In other words, it was exactly like dealing with a tester in a traditional software shop.

Dealing with testers of varying experience and skill is simply part of the job, regardless of who you're writing software for. I think there are indeed problems with the App Store review process in its current form, but, frankly, a tweet about a rejected app alone isn't news worthy. In case of Tweetie, I would expect that a problem like that could be resolved with a simple email, and I wouldn't expect any developer to have a problem with that. When a reviewer doesn't respond to follow-up emails, doesn't clarify the problem, and doesn't accept that they've made a mistake, then *that* is news, but you just don't hear about those cases very often because I don't think it happens very often.

Jeff LaMarche said...


It sounds like Tweetie had a similar experience. Loren re-submitted it without change, and it was accepted.

It's good that it ended well for you and him, but Apple really should be thinking about the repercussions of putting people that junior into a role that is public facing. It's bad decision-making by management, something that's usually pretty rare at Apple.

mike said...

I'm not pro-profanity, but pro-consistency: why is it not allowed for an app to display a naughty word, but it's fine for iTunes to sell songs with explicit lyrics?

Mostly Torn said...


I have to disagree with your comment, "When a reviewer doesn't respond to follow-up emails, doesn't clarify the problem, and doesn't accept that they've made a mistake, then *that* is news, but you just don't hear about those cases very often because I don't think it happens very often."

That happens very frequently, perhaps 50% of the time in my own experiences, and on many developer forums it is a very common complaint.

I've successfully submitted over 40 apps to the store so have had my share of rejection. Yes, sometimes the rejection letter is helpful, the reviewer may even include a screenshot, but most of the time the rejection is plain boiler-plate text and often times it isn't even clear how the boiler-text applies to the app in question.

Email replies to the reviewer (using the "follow-up number") are frequently ignored and if there is a reply, it's often just a reiteration of the same original boiler-plate text and seems to ignore requests for clarification or any specific questions that might have been posed.

My overall impression has been that the reviewer is not even willing to consider they might be making a mistake in their interpretation of the SDK or HIG.

Perhaps I am just unlucky and I'm getting the least skilled reviewers. But, if there was one word I'd use to describe the review process, it would be "arbitrary". It's terribly inconsistent.

pippin said...


as I read your article I feel I have to share some of my experience, too. And I think it may be helpful to some.


Whenever I'm calmed down enough to look at it a bit from a distance I feel it's probably not too bad a process for a task they are up to.


ScottYelich said...

Mostly Torn, have you ever had the pleasure of dealing with the USPTO? Makes Apple seem like a pleasure.

On the app side, I've had a single app rejected 7 times. Can anyone beat that?

Also, mark my words... I"m going to go for the first app to get rejected for too much functionality. :-)
(And if/when that happens, I'll go for a rejection for being to... uh... purple?)