Scope: Multiple Egnyte Instances across multiple business entities
Identified Files: Over 100 Million
Total Storage: 800-900TB

Utilising the Microsoft Azure platform Support Partners designed and developed a migration engine that not only reduced costs but also improved compliance

Due to the scale of the problem the client faced Support Partners divided the project into 2 phases. Firstly an analysis phase of the content stored in Egynte and secondly to sort and migrate the content to Microsoft Sharepoint, Microsoft OneDrive and Azure Blob storage.

The analysis enabled both Support Partners and the client understand the volume and scale of content. This included details of the type of content stored by users, how and why. By using this data, a rules engine was developed that ensured content could be migrated efficiently, securely and to the most appropriate destination .

For example, the analysis showed that there was a large number of video files (production broadcast videos) being stored on a global scale. Using the analysis data we were able to identify these files and build a rule to move them to lower cost Blob storage while moving other collaborative assets to Sharepoint maintaining the value of sharing that Egnyte offered.

A second example was centred around compliance. The analysis highlighted a large amounts of Technical files (Systems files, Disk Images) that had being stored within Egnyte. These files posed a risk to corporate compliance and security. Through the analysis, non compliant files were removed from the migration.

Analysis dashboards created included:

• Summary data of the total storage by file type and category
• Egnyte analysis progress showing how many records were processed (on average we would process 880,000 records per day with peaks around 3 millions records)
• Zero size folders (Client were able to remove 16000 empty top level folders)
• Top level folder analysis showing the breakdown of a top level folder and the sub contents (This was used to identify owners of these folders and the migration approach)

*top-level analysis dasboard

Our project approach followed these steps:

• Discovery phase – an investigation to the scope and definition of the requirements.
• Development phase 1 – analysis engine build and testing
• Analysis phase – report and dashboard development and analysis working closely with the client into the insight. Scope and definition of the requirements for the migration engine
• Development phase 2 – migration engine build and test
• Release to production with ongoing support

All the time ensuring that anything developed could be repurposed or would have value downstream.

*Real-time analysis data

*Analysis of file types

• Ability to define rules on any criteria stored within the database, for example by file category, extension, size, name, permission with field-level logic.
• Ability to choose different endpoint by rule (this could be Blob storage, Sharepoint or OneDrive)
• Ability to apply rules to specific top-level folder for migration
• Ability to copy comments to Sharepoint
• Ability to create stub file in Sharepoint for content migrated to Blob
• Ability to track the progress of each migration, identify errors, rerun failed files.
• Ability to use Microsoft Delegated Permissions from a user to access their OneDrive to migrate content to

All migrations were tracked in the database and reports were generated showing completed migrations, amount of content migrated and the time to migrate. Enabling the client to have a real-time view of the migration status.

*rules engine UI

*rules list

Technologies Used:

• Azure Functions
• Azure SQL Database
• Blob Storage
• Service Bus
• Azure App Services
• Microsoft Graph API
• Egnyte API
• Blob API
• Azure Application Insights

• It has enabled our client to understand how collaborative platforms are being used. In this case from Backups to Work in Progress repositories to an Ad hoc storage for unstructured data
• Migrating their assets out of Egnyte and onto Microsoft it saved them hundreds of thousand dollars per year in renewal and consumption costs.
• The client is now able to leverage their investment in Sharepoint and Microsoft teams for collaboration. From an end-user perspective, assets were easier to access and it was easier to store and secure newly created assets.
• Able to leverage lower-cost storage for older content and large files
• Comparable migration solutions (Off the shelf) were priced per TB, by designing and developing this engine we were able to deliver this 40% saving on the initial investment.
• The core development for the migration engine is being repurposed as a policy-based archiving solution for SharePoint
• It also ensured that our clients’ global offices and users were compliant with security policies around data storage