I wanted to provide an update and an apology.
Everything was back to normal yesterday at 5:10 PM Eastern time. The issue started about 3pm, so the disruption lasted 2 hectic hours before we were able to figure out what went wrong and how to rectify it.
Who was affected? Accounts that signed in to Method between 3pm and 5:10pm Eastern. Accounts that were already signed in and using Method likely didn't not see the problem. This only happened to some standard QuickBooks and CRM screens. Custom screens were not affected.
So, what happened? It was a failure on our side in our quality assurance (QA) process. We didn't actually do an update to the production web servers yesterday, although I know it appeared that way to affected accounts. What happened is actually a bit more complicated.
Complicated and technical overview of what happened: We've been working on an update since December that brings in features to calendars, the report designer, HTML editors, enhanced Safari and Chrome compatibility, and bunch of other big changes. It has been going through our QA process for the last month, which entails slowly moving it through QA stages. All but the final stage are on separate test environments that have no impact on the production servers. Yesterday, however, we moved the update to the final test web server which shares one resource with the production web servers: the production database server. When we published our standard Method screens on our test environment, a software bug caused the database server to get updated in a way that broke compatibility to production versions of the standard Method screens using the same database server. In hindsight, our development team looking at the code can't believe we made such a mistake....but we did. It took us about an hour to figure out what had occurred, and another hour to write, test and run a program that reversed the new changes to the database.
Our sincere apologies for the disruption. We are evaluating our QA process to see how we can tighten it to make sure this doesn't happen again.
Paul