tag:status.demodesk.com,2005:/historyDemodesk Status - Incident History2024-03-28T15:47:32ZDemodesktag:status.demodesk.com,2005:Incident/199302312024-02-07T08:00:00Z2024-02-07T12:52:12ZGDPR compliant recording issues<p><small>Feb <var data-var='date'> 7</var>, <var data-var='time'>08:00</var> UTC</small><br><strong>Resolved</strong> - Very few meetings that had the feature "GDPR compliant recording" enabled, faced a problem where participants could not submit the form to provide their consent. This was a UI defect that led a popup window to stay open after clicking "accept". Besides the misbehavior of the model, recordings did start correctly as soon as all participants gave their consent.<br /><br />The UI defect was fixed by rolling back the faulty changes.</p>tag:status.demodesk.com,2005:Incident/185670272023-09-21T16:09:22Z2023-09-21T16:09:22ZGoogle has an active incident for their Calendar API<p><small>Sep <var data-var='date'>21</var>, <var data-var='time'>16:09</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>21</var>, <var data-var='time'>14:40</var> UTC</small><br><strong>Identified</strong> - Google Calendar is having issues. Demodesk functionalities that read or write into Google Calendars might not work as expected.<br /><br />Google shares status updates and details on their status page here: https://www.google.com/appsstatus/dashboard/incidents/7uJZ5F1Uy4n1n74iMacQ</p>tag:status.demodesk.com,2005:Incident/184886292023-09-13T08:00:00Z2023-09-13T09:32:16ZIncreased loading time<p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>08:00</var> UTC</small><br><strong>Resolved</strong> - For a period of a few minutes, Demodesk experienced significantly increased loading times across the whole application.</p>tag:status.demodesk.com,2005:Incident/173403732023-05-23T16:00:42Z2023-05-23T16:00:42ZIncreased loading time<p><small>May <var data-var='date'>23</var>, <var data-var='time'>16:00</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'>23</var>, <var data-var='time'>12:20</var> UTC</small><br><strong>Monitoring</strong> - We have fixed the issue and are currently monitoring the systems.</p><p><small>May <var data-var='date'>23</var>, <var data-var='time'>12:16</var> UTC</small><br><strong>Investigating</strong> - We currently have an incident that increases loading times on all Demodesk functionalities – we're investigating the cause.</p>tag:status.demodesk.com,2005:Incident/103665692022-06-21T08:24:54Z2022-06-21T08:24:54ZDegraded system performance as our DNS and CDN Cloudflare is facing issues<p><small>Jun <var data-var='date'>21</var>, <var data-var='time'>08:24</var> UTC</small><br><strong>Resolved</strong> - Cloudflare resolved their incident and Demodesk is fully back to normal operation. Details on their outage can be found here: https://www.cloudflarestatus.com/incidents/xvs51y9qs9dj</p><p><small>Jun <var data-var='date'>21</var>, <var data-var='time'>07:34</var> UTC</small><br><strong>Monitoring</strong> - We mitigated the major impact and the core functionality our system is restored. <br /><br />Since Cloudflare is widely spread we also see errors in our communication with third party integrations (eg. Hubspot).</p><p><small>Jun <var data-var='date'>21</var>, <var data-var='time'>07:07</var> UTC</small><br><strong>Investigating</strong> - We see issues with various parts of the system. The root cause is an incident of Cloudflare (https://www.cloudflarestatus.com/).</p>tag:status.demodesk.com,2005:Incident/95754532022-03-18T15:30:37Z2022-03-18T15:52:21ZSlow and unresponsive application<p><small>Mar <var data-var='date'>18</var>, <var data-var='time'>15:30</var> UTC</small><br><strong>Resolved</strong> - Our API experienced high latencies which made it hard to use the application. Over a period of ~10 minutes, it was slow to us and for ~5 minutes most requests from the frontend timed out.<br /><br />We quickly identified the issue after the application was unresponsive and we were able to restore service within some minutes.</p>tag:status.demodesk.com,2005:Incident/94796952022-03-07T17:55:56Z2022-03-07T17:55:56ZIncreased meeting start times<p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>17:55</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>11:48</var> UTC</small><br><strong>Investigating</strong> - We are currently looking into increased meeting start times that affect a minority of meetings (<1%).</p>tag:status.demodesk.com,2005:Incident/94375932022-03-01T20:10:10Z2022-03-01T20:10:10ZTwilio outage<p><small>Mar <var data-var='date'> 1</var>, <var data-var='time'>20:10</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'> 1</var>, <var data-var='time'>20:09</var> UTC</small><br><strong>Update</strong> - Audio/video is operation again and we're continuing to monitor the situation.</p><p><small>Mar <var data-var='date'> 1</var>, <var data-var='time'>18:22</var> UTC</small><br><strong>Investigating</strong> - It's currently not possible to start meetings, as one of our service providers for audio/video communication has an outage. https://status.twilio.com/ We're working on a workaround and trying to resolve the situation as quickly as possible.</p>tag:status.demodesk.com,2005:Incident/87305732021-12-01T11:04:30Z2021-12-01T11:04:30ZDelays in parts of the system<p><small>Dec <var data-var='date'> 1</var>, <var data-var='time'>11:04</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'> 1</var>, <var data-var='time'>08:51</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we consider the issue to be solved but keep monitoring the results.<br /><br />The expected impact is that CRM synchronization was delayed or even skipped in some rare cases.</p><p><small>Dec <var data-var='date'> 1</var>, <var data-var='time'>07:48</var> UTC</small><br><strong>Identified</strong> - We identified an issue that causes certain functionality to be delayed (eg. syncing with CRMs). Demo starts should not be affected by this.<br /><br />We're working on resolving it.</p>tag:status.demodesk.com,2005:Incident/86610852021-11-22T18:10:44Z2021-11-22T18:10:44ZDelays in starting recordings and meetings<p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>18:10</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>13:59</var> UTC</small><br><strong>Monitoring</strong> - Due to issue with a third party provider, we are experiencing delays in recordings as well as a minor subset of meetings starting times.</p>tag:status.demodesk.com,2005:Incident/83938902021-11-05T16:35:48Z2021-11-05T16:35:48ZGoogle SSO login failures<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>16:35</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>11:40</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>11:05</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>10:36</var> UTC</small><br><strong>Identified</strong> - We're experiencing disruptions related to the Google SSO functionality. The issue has been identified and a fix will be deployed and fully functional soon.</p>tag:status.demodesk.com,2005:Incident/71300042021-06-01T10:11:21Z2021-06-01T10:11:21ZRecordings not accessible<p><small>Jun <var data-var='date'> 1</var>, <var data-var='time'>10:11</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'> 1</var>, <var data-var='time'>07:40</var> UTC</small><br><strong>Monitoring</strong> - DigitalOcean has fixed the issue, we're now going through all affected recordings to verify they're accessible.</p><p><small>May <var data-var='date'>31</var>, <var data-var='time'>12:14</var> UTC</small><br><strong>Investigating</strong> - Our storage provider DigitalOcean has an outage, slowing down the availability of our recordings. We are working on a fix. (DigitalOcean ticket for reference: https://status.digitalocean.com/incidents/dw9913v02mq7)</p>tag:status.demodesk.com,2005:Incident/65616552021-03-18T08:00:00Z2021-03-18T13:28:11ZErrors with booking for Microsoft Office users<p><small>Mar <var data-var='date'>18</var>, <var data-var='time'>08:00</var> UTC</small><br><strong>Resolved</strong> - Microsoft Office is currently facing internal issues affecting your Demodesk scheduling and booking pages. We will update you immediately once it is resolved.<br /><br />Google Gsuite customers are not affected by this. You can hold meetings that are already scheduled.</p>tag:status.demodesk.com,2005:Incident/65541952021-03-17T17:00:00Z2021-03-17T17:35:22ZGoogle Cloud partial network outage<p><small>Mar <var data-var='date'>17</var>, <var data-var='time'>17:00</var> UTC</small><br><strong>Resolved</strong> - Google Cloud reports increased network latencies, according to our monitoring the situation was resolved 17:20 UTC.<br /><br />Google status tracker for details: https://status.cloud.google.com/incident/cloud-networking/21006</p>tag:status.demodesk.com,2005:Incident/59978082021-01-11T18:10:24Z2021-01-11T18:10:24ZSome recordings are currently unaccessible<p><small>Jan <var data-var='date'>11</var>, <var data-var='time'>18:10</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>11</var>, <var data-var='time'>16:10</var> UTC</small><br><strong>Identified</strong> - One of our storage cloud providers is having an issue with serving files. <br />This means that viewing recordings of past meetings might not be possible currently.<br /><br />The recordings of running demos are not affected.</p>tag:status.demodesk.com,2005:Incident/58823832021-01-02T15:48:27Z2021-01-02T15:48:27ZPlaybook previews update<p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>15:48</var> UTC</small><br><strong>Completed</strong> - The storage migration of our image assets has been completed successfully and without downtime.</p><p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>14:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Dec <var data-var='date'>25</var>, <var data-var='time'>12:03</var> UTC</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time. Playbook previews may not show as expected.</p>tag:status.demodesk.com,2005:Incident/56608572020-11-26T11:29:16Z2020-11-26T11:29:16ZDelayed demo starts for some customers<p><small>Nov <var data-var='date'>26</var>, <var data-var='time'>11:29</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved and we identified specific ways to prevent this from happening again which will be implemented.</p><p><small>Nov <var data-var='date'>26</var>, <var data-var='time'>11:16</var> UTC</small><br><strong>Monitoring</strong> - We identified issues in our networking configuration and were able to fix the issues. We're currently monitoring the configuration change and identify the next steps to prevent this error in the future.</p><p><small>Nov <var data-var='date'>26</var>, <var data-var='time'>09:05</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.demodesk.com,2005:Incident/53961452020-10-23T12:38:02Z2020-10-23T12:38:02ZPlaybook previews not loading<p><small>Oct <var data-var='date'>23</var>, <var data-var='time'>12:38</var> UTC</small><br><strong>Resolved</strong> - DigitalOcean has resolved the issue.</p><p><small>Oct <var data-var='date'>23</var>, <var data-var='time'>12:15</var> UTC</small><br><strong>Identified</strong> - Our service provider DigitalOcean has identified an issue with serving files: https://status.digitalocean.com/incidents/n14x2vg77hgq<br /><br />This is currently preventing playbook previews to be loaded. It has no impact on holding your meetings.</p>tag:status.demodesk.com,2005:Incident/52250452020-10-10T12:00:29Z2020-10-10T12:00:29ZScheduled maintenance<p><small>Oct <var data-var='date'>10</var>, <var data-var='time'>12:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>10</var>, <var data-var='time'>08:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>17</var>, <var data-var='time'>08:55</var> UTC</small><br><strong>Scheduled</strong> - We're upgrading our reverse proxy and SSL termination setup and deploy a new authentication system.<br />You'll be logged out of your current session (also affects running meetings).</p>tag:status.demodesk.com,2005:Incident/52701192020-10-07T11:30:00Z2020-10-07T14:52:37ZSlow responses and timeouts for 2-5 minutes<p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>11:30</var> UTC</small><br><strong>Resolved</strong> - During a planned deployment we faced slow responses and timeouts for a small percentage of users. Besides "Demodesk feeling slow" for some users, the symptoms included misbehavior or delays of functionality (like mute/unmute, slide switches, meeting starts, ...).<br /><br />The issue was resolved within 2 to 5 minutes and we're implementing additional safeguards to prevent this situation in the future.</p>tag:status.demodesk.com,2005:Incident/52249482020-10-02T07:27:43Z2020-10-02T07:27:43ZMicrosoft Office365 outage affects some Demodesk users<p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>07:27</var> UTC</small><br><strong>Resolved</strong> - Microsoft resolved their issues. For some users, it was already resolved earlier.</p><p><small>Oct <var data-var='date'> 1</var>, <var data-var='time'>08:39</var> UTC</small><br><strong>Identified</strong> - Microsoft reports having issues with various Office365 services. We see increased error rates related to booking for Demodesk users with Office365 integrations. <br />It might also affect the "Sign in with Office 365" feature of Demodesk and outgoing emails.<br /><br />Microsoft is tracking their status here: https://admin.microsoft.com/servicestatus</p>tag:status.demodesk.com,2005:Incident/51412062020-09-26T10:00:36Z2020-09-26T10:00:36ZScheduled maintenance<p><small>Sep <var data-var='date'>26</var>, <var data-var='time'>10:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Sep <var data-var='date'>26</var>, <var data-var='time'>08:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Sep <var data-var='date'>21</var>, <var data-var='time'>09:54</var> UTC</small><br><strong>Scheduled</strong> - We're upgrading our reverse proxy and SSL termination setup. Dashboard activity and meetings should not be affected.</p>tag:status.demodesk.com,2005:Incident/50508702020-09-09T09:00:00Z2020-09-09T12:55:01ZDelayed demo starts for some customers<p><small>Sep <var data-var='date'> 9</var>, <var data-var='time'>09:00</var> UTC</small><br><strong>Resolved</strong> - We faced an issue in a regional cluster, which led to delays when starting a meeting for some customers. The issue was quickly resolved by shifting affected customers to a different cluster in the same region while we were investigating the root cause. <br />The root cause was later identified and a fix implemented.</p>tag:status.demodesk.com,2005:Incident/46215282020-07-15T16:20:01Z2020-07-15T16:49:10ZA few users experience a white page<p><small>Jul <var data-var='date'>15</var>, <var data-var='time'>16:20</var> UTC</small><br><strong>Resolved</strong> - This issue has been resolved.</p><p><small>Jul <var data-var='date'>15</var>, <var data-var='time'>16:13</var> UTC</small><br><strong>Monitoring</strong> - Resolution: Please press CTRL+F5 in your browser to resolve this issue.<br /><br />Explanation: There was a time period of 5 minutes where users that freshly accessed Demodesk couldn't load a static file. This 404 request was cached inside the browser. The root cause was a misconfiguration of the update strategy of our static file servers.</p>tag:status.demodesk.com,2005:Incident/45642612020-07-08T16:01:53Z2020-07-08T16:01:53ZScreensharing for running meetings is not working<p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>16:01</var> UTC</small><br><strong>Resolved</strong> - Screen sharing API is back up in all regions.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>15:56</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>15:55</var> UTC</small><br><strong>Identified</strong> - We are continuing to work on a fix for this issue.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>15:55</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>15:37</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>15:27</var> UTC</small><br><strong>Investigating</strong> - We're working on a workaround</p>