Three, Seven or Twelve Steps to Heaven?
“Three Steps to Heaven” was the final single released by Eddie Cochran before his untimely death by car accident in 1960. Eddie’s formula was quite simple: ‘Step one, you find a girl you love; Step two, she falls in love with you; Step three, you kiss and hold her tightly’. In August 2011, Gartner expanded Eddie’s formula to become seven steps as they sought to help organisations migrate legacy Lotus Notes applications.
Conceptually Gartner’s steps were straight forward and common sense and many organisations migrated their email systems over the years. Unfortunately, many organisations have yet to start the process of migrating their data and applications away from the Lotus Notes platform. There are many reasons as to why this is so but probably the two key ones are a truly compelling event that forces the issue and a lack of expertise. Furthermore a recent survey by international law firm Paul Hastings, most top UK and US firms are still overestimating their state of readiness.
The compelling event is the General Data Protection Regulation (GDPR), which becomes UK law on 25th May 2018. It is compelling because organisations are expected to create a single view of a customer – leaving data in old Notes databases is likely to render users non-compliant.
The proposed new EU data protection regime extends the scope of the EU data protection law with a strict data protection compliance regime with severe penalties of up to 4% of worldwide turnover. The information commissioner (ICO) in the UK has created a useful guide with twelve steps in place to help address any personal data issues that an organisation might have. In a recent blog cloud & security expert Paul Lees provides an excellent overview of the steps in clear non-technical terms. Awareness of the impact GDPR is a real issue especially with the SME marketspace. A recent Close Brothers study of 900 small and medium-sized enterprises (SMEs) from across the UK and Ireland claimed that just one in four has started preparing for it, while only one in three is aware of the implications of GDPR.
Lack of expertise is easy to fix, there are organisations that can still provide expertise in assisting organisations off of Notes, although the numbers are dwindling rapidly. However you chose to proceed it is worth finding a consultancy with a technology set that follows the same methodology as proposed by Gartner. These are the steps Lotus Notes users need to consider to help you to comply with the new law when it comes into force.
Here are three practical steps to help Notes users with GDPR compliance.
Before anything else happens you need to get a concise and accurate picture of your Notes environment. You need a process that and technology that accesses your Notes estate and provides you with a complete list of all the NSF files you have. As a result you will have a detailed breakdown of applications structure and content.
By analysing the output presented you will quickly discover which databases are affected by the new laws thus enabling you to pick which to archive and which to replace in order to remain compliant.
A key area of importance to the GDPR compliance directives is being able to get to all the data that is held on an individual so that, for example, a request to be forgotten is executed with a high degree of certainty. This means that getting data out of the Notes databases into an easily searched Relational Database is critical.
It goes without saying that there are many ways to extract data from Notes. Organisations can spend many, many man hours creating Notes scripts to extract the data. This approach is labour intensive, expensive and time consuming. The scripts themselves are straightforward enough to develop but before they are written and thoroughly tested there is an entire process of analysing the structure of the Notes database to determine what is extracted, what is ignored, how it is structured and where it is going to go. Phew, you get the idea!!
Oh, and once the IBM Lotus Notes side of it is done, you then need to design the SQL database, deploy it and write more scripts to populate it with the data. You may even need a different consultant, someone who understands SQL. This all adds time and cost to your project and to do it properly typically take several days per application just to get the data migrated. Then there is designing, testing, executing, rewriting and doing it again and again until it works.
But there is a better way of extracting the data. Technology exists that takes the majority of the manual effort away and provides a more automated and, more importantly, repeatable process. This allows you to extract the data in hours instead of months.
The most common approach that we see from our customer base is to extract all data, metadata, attachments, ACLs etc. into a Microsoft SQL or MySQL database. This enables them to automatically use the existing application “structure” to define the database. Also any attachments associated with the NSF are automatically linked and can be stored anywhere. If a database is not required then another potential automatic option is to extract the data in XML format – the schema can be adjusted to enable ingestion of the data by an application such as MS Dynamics CRM. Also it is possible to take all attachments and squirt them into a Document or Content Management System (DMS or CMS) such as Alfresco.
Part of the process of moving off of Notes and retaining data can entail building a searchable archive.
Automated archiving solutions exist that will systematically extract data (attachments, RTF’s, images, etc.) from any number of Notes Databases and make them available to view without having an application to access them, whilst at the same time mirroring the structure of the Notes application & document database.
The resulting archive structure can then be imported into a full archiving solution for future use as and when required.
The proposed new EU data protection regime extends the scope of the EU data protection law to all foreign organisations processing data of EU residents. It provides for a harmonisation of the data protection regulations throughout the EU, thereby making it easier for non-European organisations to comply with these regulations; however, this comes at the cost of a strict data protection compliance regime with severe penalties of up to 4% of worldwide revenues. As Y2K was to COBOL so GDPR is to Notes.