Papirfly Status - Incident history https://status.papirfly.com Papirfly Tue, 6 Jan 2026 07:00:00 +0000 Tactic app scheduled maintenance Type: Maintenance Duration: 3 hours Affected Components: Produce, , Place, Point, Plan, Prove, , Papirfly Cloud Services → Cloud Services → Jan 6, 07:00:00 GMT+0 - Identified - Tactic app will undergo planned maintenance to perform a necessary system upgrade. The app may be temporarily unavailable during this period. Jan 6, 07:00:00 GMT+0 - Identified - Maintenance is now in progress. Jan 6, 10:00:00 GMT+0 - Completed - Maintenance has completed successfully. Type: Maintenance

Duration: 3 hours

Affected Components: , , , , , ,

<p><small>Jan <var data-var='date'> 6</var>, <var data-var='time'>07:00:00</var> GMT+0</small><br><strong>Identified</strong> - Tactic app will undergo planned maintenance to perform a necessary system upgrade. The app may be temporarily unavailable during this period..</p> <p><small>Jan <var data-var='date'> 6</var>, <var data-var='time'>07:00:00</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress..</p> <p><small>Jan <var data-var='date'> 6</var>, <var data-var='time'>10:00:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully..</p> ]]> Tue, 6 Jan 2026 07:00:00 +0000 https://status.papirfly.com/maintenance/cmlupmr9b0hehe0oe82mejots https://status.papirfly.com/maintenance/cmlupmr9b0hehe0oe82mejots Tactic app scheduled maintenance Type: Maintenance Duration: 3 hours Affected Components: Produce, , Place, Point, Plan, Prove, Papirfly Cloud Services → Dec 11, 07:00:00 GMT+0 - Identified - Tactic app will undergo planned maintenance to perform a necessary system upgrade. The app may be temporarily unavailable during this period. Dec 11, 07:00:00 GMT+0 - Identified - Maintenance is now in progress. Dec 11, 10:00:00 GMT+0 - Completed - Maintenance has completed successfully. Type: Maintenance

Duration: 3 hours

Affected Components: , , , , ,

<p><small>Dec <var data-var='date'> 11</var>, <var data-var='time'>07:00:00</var> GMT+0</small><br><strong>Identified</strong> - Tactic app will undergo planned maintenance to perform a necessary system upgrade. The app may be temporarily unavailable during this period..</p> <p><small>Dec <var data-var='date'> 11</var>, <var data-var='time'>07:00:00</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress..</p> <p><small>Dec <var data-var='date'> 11</var>, <var data-var='time'>10:00:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully..</p> ]]> Thu, 11 Dec 2025 07:00:00 +0000 https://status.papirfly.com/maintenance/cmlupklv80f72wndf696vhktv https://status.papirfly.com/maintenance/cmlupklv80f72wndf696vhktv Global Cloudflare Outage Type: Incident Duration: 7 hours and 42 minutes Affected Components: Web Services Nov 18, 11:48:00 GMT+0 - Identified - Cloudflare is experiencing a global outage, which has caused many of our Brand Portal sites to become unstable. Nov 18, 19:30:00 GMT+0 - Resolved - Cloudflare's issues have been resolved Nov 18, 15:38:00 GMT+0 - Monitoring - We have implemented a solution to move common services away from Cloudflare, and most of our Brand Portal sites are now operational. However, sites with customer URLs that are directly configured through Cloudflare remain dependent on Cloudflare's operational status. Type: Incident

Duration: 7 hours and 42 minutes

Affected Components:

<p><small>Nov <var data-var='date'> 18</var>, <var data-var='time'>11:48:00</var> GMT+0</small><br><strong>Identified</strong> - Cloudflare is experiencing a global outage, which has caused many of our Brand Portal sites to become unstable..</p> <p><small>Nov <var data-var='date'> 18</var>, <var data-var='time'>19:30:00</var> GMT+0</small><br><strong>Resolved</strong> - Cloudflare's issues have been resolved.</p> <p><small>Nov <var data-var='date'> 18</var>, <var data-var='time'>15:38:00</var> GMT+0</small><br><strong>Monitoring</strong> - We have implemented a solution to move common services away from Cloudflare, and most of our Brand Portal sites are now operational. However, sites with customer URLs that are directly configured through Cloudflare remain dependent on Cloudflare's operational status..</p> ]]> Tue, 18 Nov 2025 11:48:00 +0000 https://status.papirfly.com/incident/cmlup8ge60onv12d40x1h2gsp https://status.papirfly.com/incident/cmlup8ge60onv12d40x1h2gsp Scheduled maintenance on Papirfly Cloud environment Type: Maintenance Duration: 2 hours Affected Components: Produce, , Place, Point, Plan, Prove, , Papirfly Cloud Services → Cloud Services → Jun 29, 17:00:00 GMT+0 - Identified - Maintenance is now in progress. Jun 29, 17:00:00 GMT+0 - Identified - We are planning for a scheduled maintenance during this time to update the AWS K8s cluster to the latest version. During this maintenance period, there may be brief disruptions in connectivity as the cluster is updated. The availability of images and video files will not be disrupted. We apologize for any inconvenience this may cause, but this maintenance is necessary to ensure the ongoing security and reliability of the K8s cluster. Jun 29, 19:00:00 GMT+0 - Completed - Maintenance has completed successfully. Type: Maintenance

Duration: 2 hours

Affected Components: , , , , , ,

<p><small>Jun <var data-var='date'> 29</var>, <var data-var='time'>17:00:00</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress..</p> <p><small>Jun <var data-var='date'> 29</var>, <var data-var='time'>17:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are planning for a scheduled maintenance during this time to update the AWS K8s cluster to the latest version. During this maintenance period, there may be brief disruptions in connectivity as the cluster is updated. The availability of images and video files will not be disrupted. We apologize for any inconvenience this may cause, but this maintenance is necessary to ensure the ongoing security and reliability of the K8s cluster..</p> <p><small>Jun <var data-var='date'> 29</var>, <var data-var='time'>19:00:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully..</p> ]]> Sun, 29 Jun 2025 17:00:00 +0000 https://status.papirfly.com/maintenance/cmluoq9wq013qv57qhdpcrssl https://status.papirfly.com/maintenance/cmluoq9wq013qv57qhdpcrssl Scheduled maintenance on Papirfly Cloud environment Type: Maintenance Duration: 1 hour Affected Components: Produce, , Place, Point, Plan, Prove, Papirfly Cloud Services → May 17, 17:00:00 GMT+0 - Identified - Maintenance is now in progress. May 17, 17:00:00 GMT+0 - Identified - We are planning for a scheduled maintenance during this time to update the AWS RDS cluster to the latest version During this maintenance period, there may be brief disruptions in connectivity as the cluster is updated. The availability of images and video files will not be disrupted. We apologize for any inconvenience this may cause, but this maintenance is necessary to ensure the ongoing security and reliability of the RDS cluster. May 17, 18:00:00 GMT+0 - Completed - Maintenance has completed successfully. Type: Maintenance

Duration: 1 hour

Affected Components: , , , , ,

<p><small>May <var data-var='date'> 17</var>, <var data-var='time'>17:00:00</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress..</p> <p><small>May <var data-var='date'> 17</var>, <var data-var='time'>17:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are planning for a scheduled maintenance during this time to update the AWS RDS cluster to the latest version During this maintenance period, there may be brief disruptions in connectivity as the cluster is updated. The availability of images and video files will not be disrupted. We apologize for any inconvenience this may cause, but this maintenance is necessary to ensure the ongoing security and reliability of the RDS cluster..</p> <p><small>May <var data-var='date'> 17</var>, <var data-var='time'>18:00:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully..</p> ]]> Sat, 17 May 2025 17:00:00 +0000 https://status.papirfly.com/maintenance/cmluombry0pb1mk9eskjej99g https://status.papirfly.com/maintenance/cmluombry0pb1mk9eskjej99g ImageVault performance issues Type: Incident Duration: 4 hours and 40 minutes Affected Components: Web Services Jan 14, 09:50:00 GMT+0 - Investigating - ImageVault has performance issues causing the site to operate very slowly. Jan 14, 10:00:00 GMT+0 - Identified - The team is working on fixing the issue and we will get back with an update as soon as possible. Jan 14, 14:30:00 GMT+0 - Resolved - All customers who have experienced the problem and have contacted us should have received a temporary solution. Jan 16, 14:00:00 GMT+0 - Postmortem - This is an update regarding the ImageVault performance. The reason to the downtime was that we have been migrated to Azure Front Door which unfortunately ImageVault was not fully compatible with. This has meant that the cache storage does not work in the same way. All customers who have experienced the problem and have contacted us should have received a temporary solution. Since yesterday (Wednesday, 15 January 2025) we also have a permanent correction developed, and there is a new version of ImageVault including this correction. We will upgrade each customer to provide the permanent solution. We are working as quickly as we can with the upgrade to ensure that everything is resolved. You will be updated continuously through your current case, but if you have questions or follow-up problems, we refer you first to our support, which you can easily reach by responding to the case or emailing [support@papirfly.com](mailto:support@papirfly.com). Type: Incident

Duration: 4 hours and 40 minutes

Affected Components:

<p><small>Jan <var data-var='date'> 14</var>, <var data-var='time'>09:50:00</var> GMT+0</small><br><strong>Investigating</strong> - ImageVault has performance issues causing the site to operate very slowly..</p> <p><small>Jan <var data-var='date'> 14</var>, <var data-var='time'>10:00:00</var> GMT+0</small><br><strong>Identified</strong> - The team is working on fixing the issue and we will get back with an update as soon as possible..</p> <p><small>Jan <var data-var='date'> 14</var>, <var data-var='time'>14:30:00</var> GMT+0</small><br><strong>Resolved</strong> - All customers who have experienced the problem and have contacted us should have received a temporary solution..</p> <p><small>Jan <var data-var='date'> 16</var>, <var data-var='time'>14:00:00</var> GMT+0</small><br><strong>Postmortem</strong> - This is an update regarding the ImageVault performance. The reason to the downtime was that we have been migrated to Azure Front Door which unfortunately ImageVault was not fully compatible with. This has meant that the cache storage does not work in the same way. All customers who have experienced the problem and have contacted us should have received a temporary solution. Since yesterday (Wednesday, 15 January 2025) we also have a permanent correction developed, and there is a new version of ImageVault including this correction. We will upgrade each customer to provide the permanent solution. We are working as quickly as we can with the upgrade to ensure that everything is resolved. You will be updated continuously through your current case, but if you have questions or follow-up problems, we refer you first to our support, which you can easily reach by responding to the case or emailing [support@papirfly.com](mailto:support@papirfly.com)..</p> ]]> Tue, 14 Jan 2025 09:50:00 +0000 https://status.papirfly.com/incident/cmluo9urw00tev57qekf0ubbz https://status.papirfly.com/incident/cmluo9urw00tev57qekf0ubbz Scheduled maintenance on Papirfly Cloud environment Type: Maintenance Duration: 1 hour Affected Components: Produce, , Place, Point, Plan, Prove, Papirfly Cloud Services → Aug 14, 21:00:00 GMT+0 - Identified - We are planning for a scheduled maintenance during this time on the AWS RDS cluster to reload CA certificates. During this maintenance period, there may be brief disruptions in connectivity as the certificates are updated. The availability of images and video files will not be disrupted. We apologize for any inconvenience this may cause, but this maintenance is necessary to ensure the ongoing security and reliability of the RDS cluster. Aug 14, 21:00:00 GMT+0 - Identified - Maintenance is now in progress. Aug 14, 21:59:00 GMT+0 - Completed - Maintenance has completed successfully. Type: Maintenance

Duration: 1 hour

Affected Components: , , , , ,

<p><small>Aug <var data-var='date'> 14</var>, <var data-var='time'>21:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are planning for a scheduled maintenance during this time on the AWS RDS cluster to reload CA certificates. During this maintenance period, there may be brief disruptions in connectivity as the certificates are updated. The availability of images and video files will not be disrupted. We apologize for any inconvenience this may cause, but this maintenance is necessary to ensure the ongoing security and reliability of the RDS cluster..</p> <p><small>Aug <var data-var='date'> 14</var>, <var data-var='time'>21:00:00</var> GMT+0</small><br><strong>Identified</strong> - Maintenance is now in progress..</p> <p><small>Aug <var data-var='date'> 14</var>, <var data-var='time'>21:59:00</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully..</p> ]]> Wed, 14 Aug 2024 21:00:00 +0000 https://status.papirfly.com/maintenance/cmluni1zm0gjjt3ii2xci3ncb https://status.papirfly.com/maintenance/cmluni1zm0gjjt3ii2xci3ncb Problem affecting search function on Papirfly on-prem web sites Type: Incident Duration: 3 hours Affected Components: Web Services Jun 11, 08:00:00 GMT+0 - Investigating - Search function unstable Jun 11, 09:00:00 GMT+0 - Monitoring - We are now monitoring the situation to ensure we pick up any other issues that may arise. Jun 11, 11:00:00 GMT+0 - Resolved - This incident has been resolved. Jun 11, 08:30:00 GMT+0 - Identified - Some sites may experience reduced performance over the next few hours as the search indexes are being rebuilt. Type: Incident

Duration: 3 hours

Affected Components:

<p><small>Jun <var data-var='date'> 11</var>, <var data-var='time'>08:00:00</var> GMT+0</small><br><strong>Investigating</strong> - Search function unstable.</p> <p><small>Jun <var data-var='date'> 11</var>, <var data-var='time'>09:00:00</var> GMT+0</small><br><strong>Monitoring</strong> - We are now monitoring the situation to ensure we pick up any other issues that may arise..</p> <p><small>Jun <var data-var='date'> 11</var>, <var data-var='time'>11:00:00</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved..</p> <p><small>Jun <var data-var='date'> 11</var>, <var data-var='time'>08:30:00</var> GMT+0</small><br><strong>Identified</strong> - Some sites may experience reduced performance over the next few hours as the search indexes are being rebuilt..</p> ]]> Tue, 11 Jun 2024 08:00:00 +0000 https://status.papirfly.com/incident/cmgf84ves01gn259n0fv0xdcf https://status.papirfly.com/incident/cmgf84ves01gn259n0fv0xdcf Service outage alert: Some Papirfly modules not working as expected Type: Incident Duration: 8 hours and 9 minutes Affected Components: , Web Services, Legacy Brand Portals → Dec 8, 17:22:00 GMT+0 - Resolved - This incident has been resolved. Dec 8, 09:15:00 GMT+0 - Identified - We've identified an issue with one of our API endpoints, and it is currently not responding as expected. Our dedicated development team is actively investigating the issue to pinpoint the root cause and implement a swift resolution. Dec 8, 09:45:00 GMT+0 - Identified - We identified a critical issue within our AWS cluster, resulting in the failure of pods to start as expected. Our technical team is actively working on diagnosing and resolving the cluster-related issue to restore normal operation. Dec 8, 09:13:00 GMT+0 - Investigating - We regret to inform you that our AWS Cloud environment is currently experiencing an unexpected outage, impacting several services crucial to our operations. The affected features include Banner, Landing Page, and sending bulk/mass emails. > **_Only customers utilizing these functionalities may be affected._** Type: Incident

Duration: 8 hours and 9 minutes

Affected Components: ,

<p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>17:22:00</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved..</p> <p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>09:15:00</var> GMT+0</small><br><strong>Identified</strong> - We've identified an issue with one of our API endpoints, and it is currently not responding as expected. Our dedicated development team is actively investigating the issue to pinpoint the root cause and implement a swift resolution..</p> <p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>09:45:00</var> GMT+0</small><br><strong>Identified</strong> - We identified a critical issue within our AWS cluster, resulting in the failure of pods to start as expected. Our technical team is actively working on diagnosing and resolving the cluster-related issue to restore normal operation..</p> <p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>09:13:00</var> GMT+0</small><br><strong>Investigating</strong> - We regret to inform you that our AWS Cloud environment is currently experiencing an unexpected outage, impacting several services crucial to our operations. The affected features include Banner, Landing Page, and sending bulk/mass emails. > **_Only customers utilizing these functionalities may be affected._**.</p> ]]> Fri, 8 Dec 2023 09:13:00 +0000 https://status.papirfly.com/incident/cmgf6glvw00kg13h9rw70cwq4 https://status.papirfly.com/incident/cmgf6glvw00kg13h9rw70cwq4 Ending support for CBC ciphers in TLS connections to Papirfly services Type: Incident Duration: 19 days and 15 hours Affected Components: , , , , Cloud Services → Legacy Services → Legacy Brand Portals → Papirfly Cloud Services → Nov 28, 15:00:00 GMT+0 - Identified - # **Announcement** On **2023-12-18 at 06:00 CET**, Papirfly will end support of cipher suites using the Cipher Block Chaining (CBC) mode of operation on hosting environments. These cipher suites are known to be susceptible to attacks such as _padding oracle attack_, which can lead to data leaks and other security issues. For a list of TLS protocols and cipher suites that will be supported on each hosting environment after the above date, please consult the relevant section below. # **How does this affect you?** ## Browser support Most modern browsers, including Google Chrome, Microsoft Edge, Mozilla Firefox and Apple Safari support strong ciphers. We recommend updating your browser to the most recent version available. ## Applications and API access Check that support for strong ciphers, such as GCM, is enabled in your application. # Cloud hosting environment (AWS) All protocols and ciphers listed under the **TLSv1.2\_2021 policy** column in the [Supported protocols and ciphers between viewers and CloudFront](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/secure-connections-supported-viewer-protocols-ciphers.html) document will be supported on our cloud hosting environment (AWS). # On-prem hosting environment (Lan-x) The protocols and corresponding cipher suites listed below will be supported in our on-prem production environment. ## Supported TLS protocols * TLSv1.2 ## Supported ECDSA ciphers * TLS\_ECDHE\_ECDSA\_WITH\_AES\_256\_GCM\_SHA384 * TLS\_ECDHE\_ECDSA\_WITH\_AES\_128\_GCM\_SHA256 ## Supported RSA ciphers * TLS\_ECDHE\_RSA\_WITH\_AES\_256\_GCM\_SHA384 * TLS\_ECDHE\_RSA\_WITH\_AES\_128\_GCM\_SHA256 # H ave any questions or need help? C ontact: [Papirfly Support](mailto:support@papirfly.com?Subject=Ending-support-for-CBC-ciphers-in-TLS-connections-to-Papirfly-services) Dec 18, 06:00:00 GMT+0 - Resolved - This is now in effect. Type: Incident

Duration: 19 days and 15 hours

Affected Components: , , ,

<p><small>Nov <var data-var='date'> 28</var>, <var data-var='time'>15:00:00</var> GMT+0</small><br><strong>Identified</strong> - # **Announcement** On **2023-12-18 at 06:00 CET**, Papirfly will end support of cipher suites using the Cipher Block Chaining (CBC) mode of operation on hosting environments. These cipher suites are known to be susceptible to attacks such as _padding oracle attack_, which can lead to data leaks and other security issues. For a list of TLS protocols and cipher suites that will be supported on each hosting environment after the above date, please consult the relevant section below. # **How does this affect you?** ## Browser support Most modern browsers, including Google Chrome, Microsoft Edge, Mozilla Firefox and Apple Safari support strong ciphers. We recommend updating your browser to the most recent version available. ## Applications and API access Check that support for strong ciphers, such as GCM, is enabled in your application. # Cloud hosting environment (AWS) All protocols and ciphers listed under the **TLSv1.2\_2021 policy** column in the [Supported protocols and ciphers between viewers and CloudFront](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/secure-connections-supported-viewer-protocols-ciphers.html) document will be supported on our cloud hosting environment (AWS). # On-prem hosting environment (Lan-x) The protocols and corresponding cipher suites listed below will be supported in our on-prem production environment. ## Supported TLS protocols * TLSv1.2 ## Supported ECDSA ciphers * TLS\_ECDHE\_ECDSA\_WITH\_AES\_256\_GCM\_SHA384 * TLS\_ECDHE\_ECDSA\_WITH\_AES\_128\_GCM\_SHA256 ## Supported RSA ciphers * TLS\_ECDHE\_RSA\_WITH\_AES\_256\_GCM\_SHA384 * TLS\_ECDHE\_RSA\_WITH\_AES\_128\_GCM\_SHA256 # H ave any questions or need help? C ontact: [Papirfly Support](mailto:support@papirfly.com?Subject=Ending-support-for-CBC-ciphers-in-TLS-connections-to-Papirfly-services).</p> <p><small>Dec <var data-var='date'> 18</var>, <var data-var='time'>06:00:00</var> GMT+0</small><br><strong>Resolved</strong> - This is now in effect..</p> ]]> Tue, 28 Nov 2023 15:00:00 +0000 https://status.papirfly.com/incident/cmgf5ke5f00t695wotorfliju https://status.papirfly.com/incident/cmgf5ke5f00t695wotorfliju Degraded performance affecting some Papirfly web sites Type: Incident Duration: 8 hours and 30 minutes Affected Components: , Legacy Brand Portals → Sep 8, 09:00:00 GMT+0 - Investigating - Some customers are reporting degraded performance and issues with some Papirfly modules. Sep 8, 09:30:00 GMT+0 - Identified - Unnusal high load on one of our database servers caused by a running import job. We are working on minimizing the impact of the degraded performance. Some Papirfly websites may experience degraded performance and issues with some Papirfly modules while this job is running. Sep 8, 11:30:00 GMT+0 - Monitoring - Still high load on the affected database server. Sep 8, 11:00:00 GMT+0 - Monitoring - Still high load on the affected database server. Sep 8, 17:30:00 GMT+0 - Resolved - Performance is back to normal. Type: Incident

Duration: 8 hours and 30 minutes

Affected Components:

<p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>09:00:00</var> GMT+0</small><br><strong>Investigating</strong> - Some customers are reporting degraded performance and issues with some Papirfly modules..</p> <p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>09:30:00</var> GMT+0</small><br><strong>Identified</strong> - Unnusal high load on one of our database servers caused by a running import job. We are working on minimizing the impact of the degraded performance. Some Papirfly websites may experience degraded performance and issues with some Papirfly modules while this job is running..</p> <p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>11:30:00</var> GMT+0</small><br><strong>Monitoring</strong> - Still high load on the affected database server..</p> <p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>11:00:00</var> GMT+0</small><br><strong>Monitoring</strong> - Still high load on the affected database server..</p> <p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>17:30:00</var> GMT+0</small><br><strong>Resolved</strong> - Performance is back to normal..</p> ]]> Fri, 8 Sep 2023 09:00:00 +0000 https://status.papirfly.com/incident/cmgf4z5b600c393eobivadjcp https://status.papirfly.com/incident/cmgf4z5b600c393eobivadjcp Scheduled performance testing Type: Maintenance Duration: 2 hours and 6 minutes Affected Components: , Legacy Brand Portals → Aug 24, 07:45:00 GMT+0 - Identified - Performance test started Aug 24, 07:00:00 GMT+0 - Identified - A performance test has been scheduled for a Papirfly service that may for a limited period affect performance for some customers. D uring the test we will constantly monitor our systems and a bort the test if major performance degradation is detected. Aug 24, 09:51:00 GMT+0 - Completed - Performance test is complete Type: Maintenance

Duration: 2 hours and 6 minutes

Affected Components:

<p><small>Aug <var data-var='date'> 24</var>, <var data-var='time'>07:45:00</var> GMT+0</small><br><strong>Identified</strong> - Performance test started.</p> <p><small>Aug <var data-var='date'> 24</var>, <var data-var='time'>07:00:00</var> GMT+0</small><br><strong>Identified</strong> - A performance test has been scheduled for a Papirfly service that may for a limited period affect performance for some customers. D uring the test we will constantly monitor our systems and a bort the test if major performance degradation is detected..</p> <p><small>Aug <var data-var='date'> 24</var>, <var data-var='time'>09:51:00</var> GMT+0</small><br><strong>Completed</strong> - Performance test is complete.</p> ]]> Thu, 24 Aug 2023 07:00:00 +0000 https://status.papirfly.com/maintenance/cmgf4ncxx00age4wz00eyzppr https://status.papirfly.com/maintenance/cmgf4ncxx00age4wz00eyzppr Problem affecting some Papirfly web sites Type: Incident Duration: 3 hours and 57 minutes Affected Components: , Legacy Brand Portals → Nov 14, 06:30:00 GMT+0 - Investigating - We are experiencing problems with our database servers in one of our production environments making some of our Papirfly websites inaccessible. Nov 14, 08:11:00 GMT+0 - Identified - Problems with the uploading of assets were found and a fix has been applied, these issues were caused by problems with our image and video processing services. Nov 14, 09:27:00 GMT+0 - Resolved - All systems operational Nov 14, 07:04:00 GMT+0 - Identified - We have identified the issue and a fix has been applied. Nov 14, 09:18:00 GMT+0 - Monitoring - There may still be some degraded performance due to these recent problems, but we are constantly monitoring to ensure we pick up any other issues that may arise. Nov 14, 07:55:00 GMT+0 - Investigating - We have detected additional issues causing degraded performance and problems with uploading assets. Nov 14, 14:00:00 GMT+0 - Postmortem - ## Summary of the problems affecting access to some Papirfly Web Sites The issues were caused by inaccessible databases and processing services in one of our production environments after a scheduled update. * 06:15: Our systems were updated automatically (during a scheduled maintenance window) which caused some databases and processing services to be inaccessible * 07:30: Problems with access to some Papirfly Web sites were detected by the first System Administrator arriving at the office * 08:00: The issue causing inaccessible databases were identified and a fix applied * 09:05: The issue causing problems with processing services were identified and a fix applied * 09:10: Systems operational with degraded performance due to the processing backlog * 10:30: Performance issues resolved and all systems operational > Note that time is in CET Type: Incident

Duration: 3 hours and 57 minutes

Affected Components:

<p><small>Nov <var data-var='date'> 14</var>, <var data-var='time'>06:30:00</var> GMT+0</small><br><strong>Investigating</strong> - We are experiencing problems with our database servers in one of our production environments making some of our Papirfly websites inaccessible..</p> <p><small>Nov <var data-var='date'> 14</var>, <var data-var='time'>08:11:00</var> GMT+0</small><br><strong>Identified</strong> - Problems with the uploading of assets were found and a fix has been applied, these issues were caused by problems with our image and video processing services..</p> <p><small>Nov <var data-var='date'> 14</var>, <var data-var='time'>09:27:00</var> GMT+0</small><br><strong>Resolved</strong> - All systems operational.</p> <p><small>Nov <var data-var='date'> 14</var>, <var data-var='time'>07:04:00</var> GMT+0</small><br><strong>Identified</strong> - We have identified the issue and a fix has been applied..</p> <p><small>Nov <var data-var='date'> 14</var>, <var data-var='time'>09:18:00</var> GMT+0</small><br><strong>Monitoring</strong> - There may still be some degraded performance due to these recent problems, but we are constantly monitoring to ensure we pick up any other issues that may arise..</p> <p><small>Nov <var data-var='date'> 14</var>, <var data-var='time'>07:55:00</var> GMT+0</small><br><strong>Investigating</strong> - We have detected additional issues causing degraded performance and problems with uploading assets..</p> <p><small>Nov <var data-var='date'> 14</var>, <var data-var='time'>14:00:00</var> GMT+0</small><br><strong>Postmortem</strong> - ## Summary of the problems affecting access to some Papirfly Web Sites The issues were caused by inaccessible databases and processing services in one of our production environments after a scheduled update. * 06:15: Our systems were updated automatically (during a scheduled maintenance window) which caused some databases and processing services to be inaccessible * 07:30: Problems with access to some Papirfly Web sites were detected by the first System Administrator arriving at the office * 08:00: The issue causing inaccessible databases were identified and a fix applied * 09:05: The issue causing problems with processing services were identified and a fix applied * 09:10: Systems operational with degraded performance due to the processing backlog * 10:30: Performance issues resolved and all systems operational > Note that time is in CET.</p> ]]> Mon, 14 Nov 2022 06:30:00 +0000 https://status.papirfly.com/incident/cmgf40tiz00ezql53v3j68ijc https://status.papirfly.com/incident/cmgf40tiz00ezql53v3j68ijc Papirfly’s response to OpenSSL 3.x vulnerability Type: Incident Nov 1, 14:00:00 GMT+0 - Resolved - # Introduction Details of a critical vulnerability in OpenSSL will be published on 1st November 2022 between 1300 and 1700 UTC \[[1](https://mta.openssl.org/pipermail/openssl-announce/2022-October/000238.html)\]. The vulnerability only affects version 3.x of OpenSSL. OpenSSL is a widely used library for encrypting network traffic. OpenSSL rates the vulnerability as critical \[[2](https://www.openssl.org/policies/general/security-policy.html)\]. Papirfly has no more information about the vulnerability than what OpenSSL itself has published. # What we have done * Reviewed all Pairfly services available on the public Internet to find any potential usage of OpenSSL 3.x * Prepared to patch any services that may be vulnerable * Prepared to take any services offline until they are patched, if they are determined to be vulnerable # What we have found * None of our services use OpenSSL 3.x # What we are doing * Waiting for OpenSSL to publish details of the vulnerability # References * \[1\] <https://mta.openssl.org/pipermail/openssl-announce/2022-October/000238.html> * \[2\] <https://www.openssl.org/policies/general/security-policy.html> Type: Incident

<p><small>Nov <var data-var='date'> 1</var>, <var data-var='time'>14:00:00</var> GMT+0</small><br><strong>Resolved</strong> - # Introduction Details of a critical vulnerability in OpenSSL will be published on 1st November 2022 between 1300 and 1700 UTC \[[1](https://mta.openssl.org/pipermail/openssl-announce/2022-October/000238.html)\]. The vulnerability only affects version 3.x of OpenSSL. OpenSSL is a widely used library for encrypting network traffic. OpenSSL rates the vulnerability as critical \[[2](https://www.openssl.org/policies/general/security-policy.html)\]. Papirfly has no more information about the vulnerability than what OpenSSL itself has published. # What we have done * Reviewed all Pairfly services available on the public Internet to find any potential usage of OpenSSL 3.x * Prepared to patch any services that may be vulnerable * Prepared to take any services offline until they are patched, if they are determined to be vulnerable # What we have found * None of our services use OpenSSL 3.x # What we are doing * Waiting for OpenSSL to publish details of the vulnerability # References * \[1\] <https://mta.openssl.org/pipermail/openssl-announce/2022-October/000238.html> * \[2\] <https://www.openssl.org/policies/general/security-policy.html>.</p> ]]> Tue, 1 Nov 2022 14:00:00 +0000 https://status.papirfly.com/incident/cmgf3k1a1005yqpjjn9va0hig https://status.papirfly.com/incident/cmgf3k1a1005yqpjjn9va0hig Network problem affecting several Papirfly web sites Type: Incident Duration: 40 minutes Affected Components: , Legacy Brand Portals → Oct 6, 13:49:00 GMT+0 - Investigating - Some Papirfly services started responding slower than usual. It appears to be caused by network instability. We are investigating together with our hosting provider. Oct 6, 14:20:00 GMT+0 - Monitoring - All Papirfly services are working normally again. We are following the situation and investigating the root cause of the earlier disruption. Oct 6, 14:29:00 GMT+0 - Resolved - All Papirfly services are working normally. Oct 7, 11:00:00 GMT+0 - Postmortem - Our hosting provider checked their network for anomalies and found signs of resource limits in a firewall. The firewall logs have been sent to the vendor for further analysis and the firewalls were upgraded and restarted during a planned maintenance window on Thursday evening. The network instability occurred between 15:48:40 and 16:20:40 CEST on Thursday 2022-10-06. Type: Incident

Duration: 40 minutes

Affected Components:

<p><small>Oct <var data-var='date'> 6</var>, <var data-var='time'>13:49:00</var> GMT+0</small><br><strong>Investigating</strong> - Some Papirfly services started responding slower than usual. It appears to be caused by network instability. We are investigating together with our hosting provider..</p> <p><small>Oct <var data-var='date'> 6</var>, <var data-var='time'>14:20:00</var> GMT+0</small><br><strong>Monitoring</strong> - All Papirfly services are working normally again. We are following the situation and investigating the root cause of the earlier disruption..</p> <p><small>Oct <var data-var='date'> 6</var>, <var data-var='time'>14:29:00</var> GMT+0</small><br><strong>Resolved</strong> - All Papirfly services are working normally..</p> <p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>11:00:00</var> GMT+0</small><br><strong>Postmortem</strong> - Our hosting provider checked their network for anomalies and found signs of resource limits in a firewall. The firewall logs have been sent to the vendor for further analysis and the firewalls were upgraded and restarted during a planned maintenance window on Thursday evening. The network instability occurred between 15:48:40 and 16:20:40 CEST on Thursday 2022-10-06..</p> ]]> Thu, 6 Oct 2022 13:49:00 +0000 https://status.papirfly.com/incident/cmgf39m1v004drsv58i4k2kl9 https://status.papirfly.com/incident/cmgf39m1v004drsv58i4k2kl9 Papirfly’s response to Log4j vulnerability CVE-2021-44228 Type: Incident Affected Components: , , Legacy Services → Legacy Brand Portals → Dec 15, 09:30:00 GMT+0 - Resolved - # Introduction On 10th December 2021, a critical vulnerability CVE-2021-44228 - referred to as _Log4Shell_ \- was disclosed. Many large software companies and online services are affected such as Amazon, Apple iCloud, Cisco, ElasticSearch, Tesla, Twitter and many more. This article summarizes the results of our investigation to date. # Description The vulnerability is caused by Apache Log4j which is a Java-based logging library used in many products and applications. The vulnerability allows an attacker to execute their own code on a remote server, a so-called Remote Code Execution (RCE) and potentially take full control of the system. Services and systems that use Apache log4j library between versions 2.0 and 2.14.1 are affected. # What have Papirfly done? We scanned our infrastructure to identify vulnerable applications and services. Here is a list of our findings and actions taken: * None of Papirfly's Internet-facing applications use Log4j directly. * We found one application that includes an older version of Log4j (pre 2.0) that is not affected by this vulnerability. * We found one application, Elasticsearch 7.12.0, that includes Log4j 2.11.1 that is affected by this vulnerability. This application is not available from the public Internet and is only sent sanitised input from the Papirfly service, so we do not believe an attacker can exploit the vulnerability against Elasticsearch. Nevertheless, out of an abundance of caution, we have taken the following actions: * 2021-12-13 09:10:00: Following advice from the authorities and the vendor, we implemented configuration changes and restarted services to immediately mitigate the vulnerability. * 2021-12-13 11:30:00: Following advice from the vendor, we upgraded to Elasticsearch 7.16.1 which disables JNDI lookups and includes a patched version of Log4j where the JndiLookup class has been removed. * 2021-12-15 08:30:00: Our hosting provider informed us that we are using a VMware component called _vCenter_ (not Internet-facing) in our on-prem production environment that uses a vulnerable version of Log4j. They are updating our vCenter server appliance from 6.7 to 7.0 U3A. * 2021-12-15 12:01:00: A further vulnerability (CVE-2021-45046) was disclosed on 14th December after it was found that the fix to address CVE-2021-44228 in Apache Log4j 2.15.0 was incomplete in certain non-default configurations. This does not affect the Papirfly infrastructure as we have upgraded to Elasticsearch 7.16.1 which mitigates the Log4j vulnerability by removing the JndiLookup class from the classpath. * 2021-12-20 10:00:00: A further vulnerability (CVE-2021-45105) was disclosed on 17th December after it was found that Log4j 2.16 was vulnerable to another DoS vulnerability. This does not affect the Papirfly infrastructure as we have upgraded to Elasticsearch 7.16.1 which provide full protection against all known CVEs. To prevent false positive alerts in vulnerability scanners that look at only the version of the Log4j dependency, and in the interest of compliance, we are in the process of upgrading to Elasticsearch 7.16.2\. This upgrades Apache Log4j2 to version 2.17.0 and retains the mitigations delivered in 7.16.1. We continue to monitor the situation and follow recommendations from the authorities and our vendors. # Conclusion We do not believe that attackers have managed to exploit the vulnerability. We have taken steps as recommended by the authorities and our vendors to further secure Papirfly services. We are continuing to investigate our exposure to this vulnerability and will provide further updates if any new risk to our users or our products is identified. We are ready to handle any incidents. # Further information * [Norwegian National Security Authority (NSM)](https://nsm.no/fagomrader/digital-sikkerhet/nasjonalt-cybersikkerhetssenter/varsler-fra-ncsc/utvidet-oppdatering-for-apache-log4j-cve-2021-44228) * [Apache Foundation Log4j mitigation methods](https://logging.apache.org/log4j/2.x/security.html) * [Elasticsearch B.V. Security Announcement](https://discuss.elastic.co/t/apache-log4j2-remote-code-execution-rce-vulnerability-cve-2021-44228-esa-2021-31/291476) Type: Incident

Affected Components: ,

<p><small>Dec <var data-var='date'> 15</var>, <var data-var='time'>09:30:00</var> GMT+0</small><br><strong>Resolved</strong> - # Introduction On 10th December 2021, a critical vulnerability CVE-2021-44228 - referred to as _Log4Shell_ \- was disclosed. Many large software companies and online services are affected such as Amazon, Apple iCloud, Cisco, ElasticSearch, Tesla, Twitter and many more. This article summarizes the results of our investigation to date. # Description The vulnerability is caused by Apache Log4j which is a Java-based logging library used in many products and applications. The vulnerability allows an attacker to execute their own code on a remote server, a so-called Remote Code Execution (RCE) and potentially take full control of the system. Services and systems that use Apache log4j library between versions 2.0 and 2.14.1 are affected. # What have Papirfly done? We scanned our infrastructure to identify vulnerable applications and services. Here is a list of our findings and actions taken: * None of Papirfly's Internet-facing applications use Log4j directly. * We found one application that includes an older version of Log4j (pre 2.0) that is not affected by this vulnerability. * We found one application, Elasticsearch 7.12.0, that includes Log4j 2.11.1 that is affected by this vulnerability. This application is not available from the public Internet and is only sent sanitised input from the Papirfly service, so we do not believe an attacker can exploit the vulnerability against Elasticsearch. Nevertheless, out of an abundance of caution, we have taken the following actions: * 2021-12-13 09:10:00: Following advice from the authorities and the vendor, we implemented configuration changes and restarted services to immediately mitigate the vulnerability. * 2021-12-13 11:30:00: Following advice from the vendor, we upgraded to Elasticsearch 7.16.1 which disables JNDI lookups and includes a patched version of Log4j where the JndiLookup class has been removed. * 2021-12-15 08:30:00: Our hosting provider informed us that we are using a VMware component called _vCenter_ (not Internet-facing) in our on-prem production environment that uses a vulnerable version of Log4j. They are updating our vCenter server appliance from 6.7 to 7.0 U3A. * 2021-12-15 12:01:00: A further vulnerability (CVE-2021-45046) was disclosed on 14th December after it was found that the fix to address CVE-2021-44228 in Apache Log4j 2.15.0 was incomplete in certain non-default configurations. This does not affect the Papirfly infrastructure as we have upgraded to Elasticsearch 7.16.1 which mitigates the Log4j vulnerability by removing the JndiLookup class from the classpath. * 2021-12-20 10:00:00: A further vulnerability (CVE-2021-45105) was disclosed on 17th December after it was found that Log4j 2.16 was vulnerable to another DoS vulnerability. This does not affect the Papirfly infrastructure as we have upgraded to Elasticsearch 7.16.1 which provide full protection against all known CVEs. To prevent false positive alerts in vulnerability scanners that look at only the version of the Log4j dependency, and in the interest of compliance, we are in the process of upgrading to Elasticsearch 7.16.2\. This upgrades Apache Log4j2 to version 2.17.0 and retains the mitigations delivered in 7.16.1. We continue to monitor the situation and follow recommendations from the authorities and our vendors. # Conclusion We do not believe that attackers have managed to exploit the vulnerability. We have taken steps as recommended by the authorities and our vendors to further secure Papirfly services. We are continuing to investigate our exposure to this vulnerability and will provide further updates if any new risk to our users or our products is identified. We are ready to handle any incidents. # Further information * [Norwegian National Security Authority (NSM)](https://nsm.no/fagomrader/digital-sikkerhet/nasjonalt-cybersikkerhetssenter/varsler-fra-ncsc/utvidet-oppdatering-for-apache-log4j-cve-2021-44228) * [Apache Foundation Log4j mitigation methods](https://logging.apache.org/log4j/2.x/security.html) * [Elasticsearch B.V. Security Announcement](https://discuss.elastic.co/t/apache-log4j2-remote-code-execution-rce-vulnerability-cve-2021-44228-esa-2021-31/291476).</p> ]]> Wed, 15 Dec 2021 09:30:00 +0000 https://status.papirfly.com/incident/cmgatp4j609eudf8zoyai0hvo https://status.papirfly.com/incident/cmgatp4j609eudf8zoyai0hvo