Over the last few weeks users of our Business Email and Enterprise Email product have been experiencing a great deal of disruption due to instability. We have been working hard to stabilise it and we felt we owed our users an update on the reasons for the instability and the progress we have made in fixing them.
About a month ago we started migrating our Business Email users from the obsolescent webmail it used, to a new interface based on OpenXchange: an enterprise grade product with numerous collaboration features. The change was fuelled by our wish to provide a more relevant, feature-full and modern product for our users.
In the past month, we have migrated over a thousand users onto the new system. Prior to starting the migration we did performance testing in order to validate our setup. Performance tests simulate user load in an approximate way. Unfortunately, these tests could not reveal all the corner cases created by real human users.
As we added users, we started seeing different kinds of scaling problems. As we fixed one issue, we uncovered new ones. In the past month, we have tuned and optimised our database backend, upgraded hardware and tuned OpenXchange. Currently, we are engaging with OpenXchange Inc. to debug subtle issues around how the database thread pool frees up connections under peak load.
We see some progress in our efforts and we're hoping to resolve the stability problems soon.
This does not seek to excuse the poor experience you have been having. It is our constant goal to provide you with the best possible product and our inability to do so is inexcusable. We want to assure you that we have been working day and night on stabilising the product and that it is currently our highest priority. We request your patience while we battle these issues and give you the high quality email service that you deserve.
Thanks for your support and cooperation,
Internet Solutions HK team
- Thursday, April 2, 2015