Incidents | Tempico Labs Incidents reported on status page for Tempico Labs https://status.itsecurity.ee/ en Homepage recovered https://status.itsecurity.ee/ Thu, 01 May 2025 14:59:57 +0000 https://status.itsecurity.ee/#d8c7bae0c7716b55d6f75557061a6ae8c6afc43c611a902d3d80a5b779e188c4 Homepage recovered Homepage went down https://status.itsecurity.ee/ Thu, 01 May 2025 14:50:51 +0000 https://status.itsecurity.ee/#d8c7bae0c7716b55d6f75557061a6ae8c6afc43c611a902d3d80a5b779e188c4 Homepage went down Billing recovered https://status.itsecurity.ee/ Sat, 08 Mar 2025 12:00:08 +0000 https://status.itsecurity.ee/#fd96755279dc2b5d2337ad6bf39947ee3451b065d1cc9a9722242dc7d02d38c7 Billing recovered Billing went down https://status.itsecurity.ee/ Sat, 08 Mar 2025 11:45:39 +0000 https://status.itsecurity.ee/#fd96755279dc2b5d2337ad6bf39947ee3451b065d1cc9a9722242dc7d02d38c7 Billing went down Git service recovered https://status.itsecurity.ee/ Wed, 26 Feb 2025 18:46:42 +0000 https://status.itsecurity.ee/#508d3bf1eed9a5eeaa272b91a4891abd62c52d68ec1b64c51d77aaa0c7d332b6 Git service recovered Git service went down https://status.itsecurity.ee/ Wed, 26 Feb 2025 18:40:45 +0000 https://status.itsecurity.ee/#508d3bf1eed9a5eeaa272b91a4891abd62c52d68ec1b64c51d77aaa0c7d332b6 Git service went down Billing recovered https://status.itsecurity.ee/ Thu, 09 Jan 2025 13:00:15 +0000 https://status.itsecurity.ee/#29ab88815f5e7f39cafed9a868430c31262dc737c1e1bbaf6e61959a45f07619 Billing recovered Billing went down https://status.itsecurity.ee/ Thu, 09 Jan 2025 12:40:52 +0000 https://status.itsecurity.ee/#29ab88815f5e7f39cafed9a868430c31262dc737c1e1bbaf6e61959a45f07619 Billing went down There is something not quite right with our load balancers https://status.itsecurity.ee/incident/415354 Sat, 17 Aug 2024 20:09:00 -0000 https://status.itsecurity.ee/incident/415354#3080acad771a87939b8e71d97ccab202e36931ec357ad265fb6585aa26d2b705 Our engineers have applied a network fix to mitigate the issue detected. We will monitor efficiency of the fix meanwhile. There is something not quite right with our load balancers https://status.itsecurity.ee/incident/415354 Sat, 17 Aug 2024 20:09:00 -0000 https://status.itsecurity.ee/incident/415354#3080acad771a87939b8e71d97ccab202e36931ec357ad265fb6585aa26d2b705 Our engineers have applied a network fix to mitigate the issue detected. We will monitor efficiency of the fix meanwhile. There is something not quite right with our load balancers https://status.itsecurity.ee/incident/415354 Sat, 17 Aug 2024 20:09:00 -0000 https://status.itsecurity.ee/incident/415354#3080acad771a87939b8e71d97ccab202e36931ec357ad265fb6585aa26d2b705 Our engineers have applied a network fix to mitigate the issue detected. We will monitor efficiency of the fix meanwhile. There is something not quite right with our load balancers https://status.itsecurity.ee/incident/415354 Sat, 17 Aug 2024 20:09:00 -0000 https://status.itsecurity.ee/incident/415354#3080acad771a87939b8e71d97ccab202e36931ec357ad265fb6585aa26d2b705 Our engineers have applied a network fix to mitigate the issue detected. We will monitor efficiency of the fix meanwhile. There is something not quite right with our load balancers https://status.itsecurity.ee/incident/415354 Sat, 17 Aug 2024 20:00:00 -0000 https://status.itsecurity.ee/incident/415354#0a807462169c1a6948a45c1eb52655a8e9c80e5f08b6e7d6cfae5e09940a5eb4 We are investigating this matter and going to update shortly. There is something not quite right with our load balancers https://status.itsecurity.ee/incident/415354 Sat, 17 Aug 2024 20:00:00 -0000 https://status.itsecurity.ee/incident/415354#0a807462169c1a6948a45c1eb52655a8e9c80e5f08b6e7d6cfae5e09940a5eb4 We are investigating this matter and going to update shortly. There is something not quite right with our load balancers https://status.itsecurity.ee/incident/415354 Sat, 17 Aug 2024 20:00:00 -0000 https://status.itsecurity.ee/incident/415354#0a807462169c1a6948a45c1eb52655a8e9c80e5f08b6e7d6cfae5e09940a5eb4 We are investigating this matter and going to update shortly. There is something not quite right with our load balancers https://status.itsecurity.ee/incident/415354 Sat, 17 Aug 2024 20:00:00 -0000 https://status.itsecurity.ee/incident/415354#0a807462169c1a6948a45c1eb52655a8e9c80e5f08b6e7d6cfae5e09940a5eb4 We are investigating this matter and going to update shortly. Cloudflare not available in Eastern US and Western Europe https://status.itsecurity.ee/incident/387166 Thu, 20 Jun 2024 21:23:00 -0000 https://status.itsecurity.ee/incident/387166#85da7979ef392cb0f69eee2e18f9dc36becba065a9419fa515c022a12aaf4dbc Cloudflare reported the resolution. Cloudflare not available in Eastern US and Western Europe https://status.itsecurity.ee/incident/387166 Thu, 20 Jun 2024 19:25:00 -0000 https://status.itsecurity.ee/incident/387166#0d3489176a424e4eeb97f72d55c74c73364295cb82bb26203b13fa8bb8c89483 We're currently observing an increased number of monitoring probe failures due to Cloudflare having connectivity issues in various regions. Customers with projects making use of Cloudflare, or components that rely on Cloudflare may encouter significant downtime in the affected regions. Please reach our support in case if migration from Cloudflare is deemed necessary. More details: https://www.cloudflarestatus.com/incidents/p7l6rrbhysck Services, who do not use Cloudflare and as well all Tempico Labs API services are not affected and currently operational. CI builds may temporarily fail if project relies on Cloudflare. Email cluster maintenance https://status.itsecurity.ee/incident/278135 Fri, 27 Oct 2023 12:00:00 +0000 https://status.itsecurity.ee/incident/278135#fd10727ca82093fbbc6e80de5a34faec01c2d7ca480039433d18601facd03fb4 Maintenance completed Email cluster maintenance https://status.itsecurity.ee/incident/278135 Thu, 26 Oct 2023 17:30:00 -0000 https://status.itsecurity.ee/incident/278135#b40bcfb272854c4158a3910755328b1fa41f02bf2cb2046839cf5c0fa9a48933 The email cluster upgrade is planned, which may cause delays in notification emails sent and slightly affect requests sent to support email. Switch fault affecting datacenter DE3 https://status.itsecurity.ee/incident/269635 Sat, 07 Oct 2023 21:02:00 -0000 https://status.itsecurity.ee/incident/269635#fb19bcdae7ebbb90c3c61ec175d8a1ce76e425028556342f3bddaf2338e6ab82 The faulty switch has been replaced and nodes affected by the downtime are up again. Switch fault affecting datacenter DE3 https://status.itsecurity.ee/incident/269635 Sat, 07 Oct 2023 21:02:00 -0000 https://status.itsecurity.ee/incident/269635#fb19bcdae7ebbb90c3c61ec175d8a1ce76e425028556342f3bddaf2338e6ab82 The faulty switch has been replaced and nodes affected by the downtime are up again. Switch fault affecting datacenter DE3 https://status.itsecurity.ee/incident/269635 Sat, 07 Oct 2023 21:02:00 -0000 https://status.itsecurity.ee/incident/269635#fb19bcdae7ebbb90c3c61ec175d8a1ce76e425028556342f3bddaf2338e6ab82 The faulty switch has been replaced and nodes affected by the downtime are up again. Switch fault affecting datacenter DE3 https://status.itsecurity.ee/incident/269635 Sat, 07 Oct 2023 20:42:00 -0000 https://status.itsecurity.ee/incident/269635#b0a3cff9df04d0cc3382b81257d7f77fbbcf7cdc8e908cde79b127448caac6f0 We are currently experiencing a fault on our switch located in the DC DE3. Our technicians are working to recover network operations as soon as possible. Switch fault affecting datacenter DE3 https://status.itsecurity.ee/incident/269635 Sat, 07 Oct 2023 20:42:00 -0000 https://status.itsecurity.ee/incident/269635#b0a3cff9df04d0cc3382b81257d7f77fbbcf7cdc8e908cde79b127448caac6f0 We are currently experiencing a fault on our switch located in the DC DE3. Our technicians are working to recover network operations as soon as possible. Switch fault affecting datacenter DE3 https://status.itsecurity.ee/incident/269635 Sat, 07 Oct 2023 20:42:00 -0000 https://status.itsecurity.ee/incident/269635#b0a3cff9df04d0cc3382b81257d7f77fbbcf7cdc8e908cde79b127448caac6f0 We are currently experiencing a fault on our switch located in the DC DE3. Our technicians are working to recover network operations as soon as possible. Email cluster maintenance https://status.itsecurity.ee/incident/225258 Wed, 28 Jun 2023 03:00:00 +0000 https://status.itsecurity.ee/incident/225258#464b547e91f8f0859834a2bac0294b0fbd1baceefb7fc7af14473b50497a1601 Maintenance completed Email cluster maintenance https://status.itsecurity.ee/incident/225258 Tue, 27 Jun 2023 15:00:00 -0000 https://status.itsecurity.ee/incident/225258#3b3adc7d07256de70e3f5ad40ee4f89de91738cfe2b8091fe5bedab2eeefe15c The email cluster upgrade is planned, which may cause delays in notification emails sent and slightly affect requests sent to support email. Git service migration https://status.itsecurity.ee/incident/187420 Tue, 21 Mar 2023 14:58:03 +0000 https://status.itsecurity.ee/incident/187420#fa02e53606d35cfca152c57c4309a89dfde021b1c8f9c3f084e57660eb4718b0 Maintenance completed Git service migration https://status.itsecurity.ee/incident/187420 Tue, 21 Mar 2023 14:58:03 -0000 https://status.itsecurity.ee/incident/187420#9469c8156eb0f291ece2082ba1bdcd6f819812ec90cae70a6c122ed2e8c0c54d Git service will be migrated to recently added DE3 datacenter. IP address change is expected after migration is complete. The FQDN of git service would remain the same. Git service migration https://status.itsecurity.ee/incident/187420 Tue, 21 Mar 2023 10:30:00 -0000 https://status.itsecurity.ee/incident/187420#9469c8156eb0f291ece2082ba1bdcd6f819812ec90cae70a6c122ed2e8c0c54d Git service will be migrated to recently added DE3 datacenter. IP address change is expected after migration is complete. The FQDN of git service would remain the same. Maintenance on git service https://status.itsecurity.ee/incident/150978 Sat, 07 Jan 2023 00:00:00 +0000 https://status.itsecurity.ee/incident/150978#b0cb47c62194a1f0c6b1667f1438a34158941174e6495b23e53bfcff3624d942 Maintenance completed Maintenance on git service https://status.itsecurity.ee/incident/150978 Thu, 05 Jan 2023 18:00:00 -0000 https://status.itsecurity.ee/incident/150978#d8cc9f568584b01354f92432baa07a683186a3721014c9cd27c317de4bf81b2d A database maintenance of git service is scheduled. To prevent data corruption, all public access to git will be limited until maintenance is complete. Email cluster maintenance https://status.itsecurity.ee/incident/131791 Wed, 02 Nov 2022 04:00:00 +0000 https://status.itsecurity.ee/incident/131791#6790f899e1b8bf21dfb81b0d3916420ea0183df3e0811fbd436ed9e8efbb386f Maintenance completed Email cluster maintenance https://status.itsecurity.ee/incident/131791 Tue, 01 Nov 2022 18:00:00 -0000 https://status.itsecurity.ee/incident/131791#b7cfc4e28d0c529dc2ccda283a59dd14186ea58b67485659965ffa7f1d0edc01 The email cluster upgrade is planned, which may cause small delays in notification emails sent. Something is not quite right with billing calculation https://status.itsecurity.ee/incident/113253 Sat, 27 Aug 2022 22:00:00 -0000 https://status.itsecurity.ee/incident/113253#95948d32a96c8bba14dc38620f1accd92ac4ca9a029db838e977ebc324908d7b Billing calculation and real-time balance indication features are back! Something is not quite right with billing calculation https://status.itsecurity.ee/incident/113253 Mon, 22 Aug 2022 23:00:00 -0000 https://status.itsecurity.ee/incident/113253#ceb7e2f802dfcb90ef7ed9a219fd4319e9291e0ce3132cc077315e715fcb91cc We've detected double-charges for some infrastructure-related services (container service & network load balancers) and stopped real-time updates of certain billing sub-systems until situation is carefully investigated. To secure your funds, credit card auto top ups are disabled too. Maintenance on git service https://status.itsecurity.ee/incident/111712 Mon, 22 Aug 2022 14:30:00 +0000 https://status.itsecurity.ee/incident/111712#da0b25330031678696334356e84c2bde49712b8054556fa04b8332922f061743 Maintenance completed Maintenance on git service https://status.itsecurity.ee/incident/111712 Mon, 22 Aug 2022 13:30:00 -0000 https://status.itsecurity.ee/incident/111712#e7c6083949e7cfc1a328a17f3551d866698a8e07f201a05ce215aa1cd7e3ef2a A database maintenance of git service is scheduled. To prevent data corruption, all public access to git will be limited until maintenance is complete. Nomad cluster update https://status.itsecurity.ee/incident/110172 Tue, 16 Aug 2022 21:26:00 +0000 https://status.itsecurity.ee/incident/110172#8728e773dc459712f70d22388b4782c04cf2d3ca2faa0e5f026969769b16201d Maintenance completed Nomad cluster update https://status.itsecurity.ee/incident/110172 Tue, 16 Aug 2022 21:26:00 +0000 https://status.itsecurity.ee/incident/110172#8728e773dc459712f70d22388b4782c04cf2d3ca2faa0e5f026969769b16201d Maintenance completed Nomad cluster update https://status.itsecurity.ee/incident/110172 Tue, 16 Aug 2022 20:26:00 -0000 https://status.itsecurity.ee/incident/110172#cf13bce85c9c8074268fe6c821833941fffa8892f3eca601cf3b4954f5992ad7 Nomad cluster will be updated to latest version with subsequent Raft update from 2 to 3 Nomad cluster update https://status.itsecurity.ee/incident/110172 Tue, 16 Aug 2022 20:26:00 -0000 https://status.itsecurity.ee/incident/110172#cf13bce85c9c8074268fe6c821833941fffa8892f3eca601cf3b4954f5992ad7 Nomad cluster will be updated to latest version with subsequent Raft update from 2 to 3 LM1 datacentre network maintenance https://status.itsecurity.ee/incident/97825 Sun, 26 Jun 2022 12:00:00 +0000 https://status.itsecurity.ee/incident/97825#9338a229345e9d0edc2ab34619afff76ce1360d7f806d06f79dad757c091b3ed Maintenance completed LM1 datacentre network maintenance https://status.itsecurity.ee/incident/97825 Sat, 25 Jun 2022 09:00:00 -0000 https://status.itsecurity.ee/incident/97825#9c149fcaec5e101ab485ffb219e804ef4b77198ba36aa185534ae60fbbe74815 Backbone maintenance planned which may cause some non-production services and staging environments temporarily unavailable. Message broker migration https://status.itsecurity.ee/incident/89732 Fri, 06 May 2022 20:00:00 +0000 https://status.itsecurity.ee/incident/89732#1c1b5f935087d1611bdc17029afbd7b58b70a099d5eb94e9c07c925a90c6f54f Maintenance completed Message broker migration https://status.itsecurity.ee/incident/89732 Fri, 06 May 2022 15:00:00 -0000 https://status.itsecurity.ee/incident/89732#8b37194816e66b664f464ff0c7977740ac497852d0545704fb09fae9d155a297 The platform will be migrated to a new queue management system. We are expecting a small downtime (up to 5 minutes) during which certain public endpoints will not be available. Billing system maintenance https://status.itsecurity.ee/incident/80113 Mon, 07 Mar 2022 16:00:00 +0000 https://status.itsecurity.ee/incident/80113#5017d3806a4fff0b12ed4bb4089768eaaf19c3756b94fce381218273f78abb0c Maintenance completed Billing system maintenance https://status.itsecurity.ee/incident/80113 Sun, 06 Mar 2022 10:00:00 -0000 https://status.itsecurity.ee/incident/80113#f8caacf6d2a09b9ba400fdcfcd81aee3c43242f03fe40aa202ee08700ae29702 The billing system will be upgraded to a new database schema to allow better scalability and new features. Any account top up and charges will be processed automatically once maintenance is completed.