Back in August 2007, I wrote the following about the XBLA 1st party (aka MS-published) title review process:
I’ve put systems in place to hopefully help reduce the risk of my own tastes (or lack of vision) from polluting the portfolio. I can’t really discuss the details, but they include a sort of “wisdom of crowds” feedback loop, in which indie submissions are screened and rated by a group of my colleagues within Microsoft (who are asked NOT to discuss the submissions with each other before rating them — mainly to avoid group-think.) The wisdom of crowds can make my forecasts more accurate, and it can help compensate for any subconscious biases I have. Unfortunately, what I don’t believe it can do is help me identify future mega-hits (i.e. “the next Geo Wars“.)
The process I referred to in that post actually kick-started in March of 2007, so it’s been about 18 months since I started changing the way that the XBLA team reviews incoming submissions from independent developers. If you had asked me way back then when I expected to be able to judge the results of the process change, I would have said “a year from now, at the latest.” Turns out, I would have been way off. A year and a half later, only six games that we have reviewed under the new process have launched on Marketplace. Many of the games greenlit in the few months following March ’07 are still in development and/or finally nearing release.
