System Status

(7:30 AM) All system back. There was a major data center outage that latest between 2:30 AM EST and 6:30 AM EST this morning, 2/22/2010. The secondary data centers were online and working, however due to a DNS configuration error, were not picking up the traffic resulting in failures to deliver media files, RSS feeds, web players, allow logins, or uploading media.

A core aspect of our service was unavailable for 4 hours- and we do not take these issues lightly.  Our backup and failover process will be thoroughly reviewed and revised as needed today to ensure this double failure does not happen again.

(6:00 AM) Partial connectivity has been restored. A router failure at the primary data center had caused the majority of issues experienced over the last few hours. The secondary datacenters are available however traffic is not going to that failover. The team is working on the issues.

(3:17 AM) The support team is working with the data center regarding a hardware and network related issue. All services are down at this point

(2:30 AM) Issues with media delivery and web players. Support team being notified

Update 5:40 PM EST

CDN issues should be resolved. Please let us know if you have any questions or problems.

(1:15pm) Some users have been reporting slow downloads which may timeout. We have opened a ticket with our CDN to look into the issue. Please contact us at support@libsynpro.com with any questions or problems.

(8:00am) The team is working on server issues which are causing issues logging in and uploading new episodes

(9:30am) Most of the systems are back up to normal working state, there may be some issues with cached RSS feeds which will be resolved shortly as the system recycles and comes back up with new copies of all cached feeds.

(10:00) All system back to full normal operation

LibsynPRO Stats

Stats processing has been temporarily paused while we perform maintenance on the system. Please direct concerns to support@libsynpro.com.

LibsynPRO Advertising

Advertising has been temporarily paused while we run some maintenance on the system. Please direct concerns to support@libsynpro.com.

The issue has been resolved. Thanks for your patience, and please contact support@libsynpro.com if you have any questions.

There are some lingering issues from the recent update that are affecting the Wizzard Player. Item images continue to be incorrect for some users. The player is now operating with some new requirements that are causing the player to not be able to play media on certain sites. These issues are known bugs that we are investigating.

libsynPRO Application libsynPRO Uploads

Access to LibsynPro and uploads are currently disabled while we release LibsynPro3.

Our apologies for access to LibsynPro still being currently disabled. The update is taking a little longer than expected. We hope for access to be reenabled in the next couple of hours. Thank you for your patience.

We are testing and validating the data post conversion. We hope for access to be available this evening. Thank you for your continued patience.

We have brought a few networks back online while performing final testing. We hope to turn all networks back on in the next couple hours.

The update is complete and the system is nearly back to normal. There were a few players and feeds that were temporarily broken. These are issues that we're working out now.

Some unforseen caching issues has caused further delays. The team is working hard to find the bug and get everyone back online asap.

Having issues with content uploads, ad campaigns, and episode thumbnails. Running campaigns are temporarily paused while we work out the issues. There may be some errant thumbnail issues as well. The team is working to get everything right.

Upload issues resolved, campaigns back on, mixed up images are being fixed. The biggest issue at the moment is the feed caching problems which are making RSS feed urls timeout as well as the wizzard flash players. Team is hard at work to resolve all issues.

Feed caching continues to cause problems which are making RSS feed urls take a very long time to load and wizzard flash players to timeout or have very long load times. Networks are being brought on one at a time to mitigate load issues while the caches warm up. If you have immediate publishing needs, please contact support@libsynpro and we can bring your network online. We appreciate your continued patience through this transition.

Access has been restored, and all uploading and publishing features have been brought back online. Please contact support@libsynpro.com if you have any questions or problems. We appreciate your patience.

(3:00pm) We have received reports of errors when uploading media files. The issue was caused by one of our service providers. Our developers have opened a ticket with the service provider, and we will post updates on the situation as it progresses.

(3:30pm) The issue with the service provider has been resolved, and uploads are now working. We apologize for any inconvenience this may have caused – please email support@libsynpro.com if you have any questions.

(10:00pm) We received short notice from a service provider that will be performing system maintenance tonight. We will be temporarily disabling logins between the hours of 12am and 1am, and we will be providing updates on the situation as it progresses.

(1:00am) Logins reenabled - please email support@libsynpro.com if you have any questions.

(7:30pm) We've noticed some intermittent issues pushing new files to the CDN. We've contacted them and are working towards a resolution.

(9:39pm) We are still in communication with the CDN. They are still investigating the outage, and we are working with them to find a resolution as quickly as possible

(12:15am) Our CDN had a failure at one of their datacenters, which lead to some problems uploading new files for some users. This issue should be resolved and uploads are succeeding. We are running some scripts to try to catch any files that failed to upload so that we can try to reupload them. Any new uploads should succeed. If you have any problems please contact us at support@libsynpro.com.